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SECTION  r 
SUMMARY 


In  this  study  of  multisensor  displays,  the  objectives  were  to  consider  different 
display  formats,  and  to  suggest  simple,  efficient  ways  of  implementing  elec- 
tronic and  optical  methods  of  combining  images.  The  study  is  part  of  a more 
general  U.  S.  Navy  program  to  develop  a system  for  combining  the  informa- 
tion from  several  sensors  on  a single  display  surface. 

Until  recently,  the  complexity  and  number  of  cockpit  instruments  increased 
with  the  complexity  of  aircraft  and  the  functions  for  which  they  could  be  used. 
Then,  because  of  advances  in  display  technology,  multifunctional,  integrated 
display  systems  were  developed.  Their  immediate  appeal  is  that  they  require 
relatively  little  cockpit  panel  space.  However,  there  are  other,  perhaps 
more  positive,  potential  uses  of  multifunctional  displays.  It  is  by  exploring 
these  potentialities  that  we  are  likely  to  be  led  to  new  cockpit  instrument 
panel  guidelines. 

Our  main  concerns  in  presenting  images  from  a number  of  sources  are  for 
clarity  of  information  and  speed  of  assimilation.  A consideration  of  image 
content  shows  that  there  can  be  a wide  range  of  relationships  between  two  or 
more  images  that  are  to  be  presented  together.  The  relationship  is  deter- 
mined by  the  pointing  angle  of  the  sensors  delivering  the  imagery,  by  the 
geometry  of  the  images,  and  by  the  fields  of  view /magnifications  being  used. 

Imagery  from  several  sources  can  be  used  in  several  ways.  First,  an  ob- 
server may  need  to  compare  the  outputs  of  two  or  more  sources  to  derive  a 
kind  of  information  composite.  Second,  an  enhanced,  composite  image 


might  be  produced  by  combining  separate  images  on  the  observer's  screen. 
Third,  the  observer  may  need  to  monitor  information  from  more  than  one 
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source.  Fourth,  transitions  from  one  image  source  to  another  may  be 
facilitated  by  some  form  of  multiple -image  presentation. 

Eight  methods  of  presenting  multiple  images  were  identified.  They  included 
superimposition,  juxtaposition  in  time,  split  screen,  and  various  forms  of 
insetting.  The  appropriateness  of  these  eight  methods  for  each  of  the  four 
uses  of  multiple -image  presentation  was  considered.  Of  the  32  combinations 
of  uses  and  methods,  11  were  judged  to  be  promising,  a further  11  were  pos- 
sible, while  the  remaining  10  were  inappropriate. 

In  considering  implementation  we  considered  a number  of  image  sources  and 
several  cases  of  multiple -image  presentations.  They  are  listed  in  Table  1-1 
along  with  other  important  factors. 

Various  sensors  and  displays,  as  well  as  techniques  for  image  combining, 
scan  conversion  and  symbol  writing  were  reviewed.  They  were  evaluated  in 
terms  of  their  applicability  to  the  multi-image  presentation  cases  listed 
above.  Of  eight  image-combining  techniques  evaluated,  the  one  with  the 
greatest  potential  for  a low-cost,  near-term,  multi-function  display  was 
judged  to  be  optical  image  combining.  Digital  computer  techniques  for  more 
complex  applications  were  also  discussed. 

Further  research  and  development  work  is  recommended.  However,  in  both 
cases,  it  should  be  directed  towards  more  specific  applications.  The  follow- 
ing tasks  are  necessary: 

F or  research 

• Prioritization  of  four  uses  of  multi-image  presentation 

• Prepare  and  conduct  experiments  and  analyze  data  to 
determine  the  most  appropriate  method(s)  of  presenting 
multiple  images  for  highest  priority  use. 
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Table  1-1.  Requirements  for  Candidate  Image- 
Combining  Concepts 

Sensors 

• PPI  radar  with  slow  frame  rate 

• FLIR  and  LLLTV  with  high  quality  imagery 

Annotation 

• Moving  map 

• Symbols  and  graphics 

Image  Combination  Laser 

• Radar  and  moving  map 

• Inset  radar  imagery  into  E-O  imagery 

• FLIR  imagery  inset  into  LLTV  imagery 

• Radar,  FLIR,  and  LLLTV  imagery  sequentially  or 
simultaneously 

• Annotation  for  either  radar  or  E-O  imagery 
Cockpit  Environment 

• High  ambient  illumination 

• Limited  head  motion  requirement 

• Severe  vibration 

Display 

• Minimum  instrument  panel  space 

• High  visibility 

• Head-down  application  assumed 

Design 

• Simple 


• Reliable 

• Low  cost 


For  development 


• Definition  of  mission  to  establish  specific  system  requirements 

• Definition  of  system  components  for  image  combining  by 
means  of  performance  tradeoffs 

• Fabrication  and  evaluation  of  demonstration  system 

We  are  dealing  with  complex  tasks  and  techniques.  It  will  not  be  easy  to  dis- 
cover the  optimal  ways  of  combining  images.  However,  research  such  as 
that  suggested  should  help  in  defining  new  cockpit  design  guidelines. 
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SECTION  II 
INTRODUCTION 


OBJECTIVES 

This  study,  part  of  a more  general  program,  is  to  develop  a system  for 
combining  the  information  from  several  sensors  on  a single  display  surface. 
The  objectives  of  the  current  study  were  to  consider  different  display  formats, 
and  to  suggest  simple,  efficient  ways  of  implementing  electronic  and  optical 
methods  of  combining  images. 

ORGANIZATION 

There  are  five  main  sections  to  this  report.  The  first  is  an  Introduction. 

The  second.  Multi  Image  Presentation,  discusses  ways  of  combining  images, 
and  describes  those  likely  to  be  most  useful.  The  third  section,  Imple- 
mentation, details  available  equipment.  It  also  contains  a general  descrip- 
tion of  ways  various  image  combinations  might  be  achieved,  and  this  is 
followed  by  some  illustrative  examples.  The  fourth  section  is  an  Evaluation. 
The  final  section,  Recommendations,  presents  conclusions  and  recom- 
mendations for  further  research  and  development. 

This  introductory  section  deals  with  the  reasons  for  considering  multifunc- 
tional, multisensor  displays,  describes  the  cockpit  environment  in  which 
they  may  be  used,  and,  finally,  discusses  the  potential  advantages  that  might 
be  obtained  by  using  these  displays. 
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WHY  USE  MULTISENSOR  DISPLAYS? 


From  the  early  1900s  until  recent  times,  there  was  an  obvious  trend  in  the 
way  information  was  displayed  in  cockpits.  Gradually,  aircraft  became  more 
complex,  and  could  be  used  in  an  increasing  number  of  circumstances.  There 
was  a corresponding  increase  in  the  number,  and  in  the  complexity,  of  dis- 
plays and  dials.  This  trend  could  have  continued  until  pilots  were  faced  with 
an  unmanageable  multitude  of  display  surfaces. 

Fortunately,  there  are  alternative  methods  of  handling  information.  By 
using  storage  and  processing  devices,  the  trend  has  been  arrested.  Multi- 
fractional,  integrated  cockpit  display  systems  have  been  developed.  Examples 
are  provided  by  the  U.  S.  Navy's  Advanced  Integrated  Modular  Instrumentation 
System  (A1MIS)  and  Vertical  Take  Off  and  Landing  System  (VTOL),  by  the 
U.  S.  Air  Force  Digital  Avionics  Information  System  (DAIS)  and  the  B-52 
Electro-optical  Viewing  System  (EVS). 

At  present,  there  is  a transitional  stage.  This  seems  certain  to  be  followed 
by  a period  in  which  there  is  an  increasing  emphasis  on  integrated,  multi- 
functional displays. 

The  cockpit  designer's  problem  was  how  to  squeeze  another  dial,  or  display, 
into  the  already  cluttered  array  facing  the  operator.  Now,  the  designer  has 
tools,  developed  through  computer  science  and  advances  in  display  tech- 
nology, that  allows  him/her  to  cut  through  the  design  constraints  of  conven- 
tional cockpit  displays.  However,  it  may  be  that  he /she  is  wielding  a double- 
edged  sword.  These  constraints  provide  some  of  the  basic  guidelines  of 
cockpit  design.  In  eliminating  them,  the  designer  will  find  that  he/she  has 
excised  many  of  the  guidelines  as  well. 
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The  immediate  appeal  of  multifunctional,  multisensor  displays  is  that  they 
are  economical.  One  screen  is  less  expensive  than  two,  or  three,  and,  more 
importantly,  it  takes  less  space.  However,  the  use  of  these  displays  opens 
up  many  other  possibilities. 

Multisensor  displays  may  improve  both  the  quality,  and  flow  of  information. 

By  combining  different  sensor  outputs,  it  may  be  possible  to  enhance  those 
features  of  the  resultant,  composite  image  that  are  relevant  to  the  operator's 
tasks.  In  addition,  with  a multisensor  display,  the  operator  may  be  able  to 
make  smoother  transitions  between  one  sensor  output  and  another. 

These  potential  improvements  in  information  could,  in  turn,  lead  to  improve- 
ments in  a number  of  the  operator's  tasks:  navigation,  piloting,  target 
acquisition,  tracking  and  battle  damage  assessment.  These  tasks  might 
well  be  carried  out  more  rapidly,  with  more  accuracy,  and/or  with  less 
effort  by  the  operator. 

There  are  several  ways  of  combining  images:  multiple  screens,  juxtaposition 
in  time,  overlay,  split  screen  and  various  forms  of  insetting.  Investigations 
will  have  to  be  conducted  to  discover  which  techniques  produce  the  optimal 
performance  in  various  operating  tasks.  It  is  through  such  investigations 
that  new  cockpit  design  guidelines  will  appear. 

We  can  expect  the  range  of  aircraft  functions  and  activities  to  continue  to 
increase  in  the  future.  Spatial  and  financial  economy  are  important.  But, 
another  consideration  will  outweigh  them  soon,  if  is  has  not  already.  We 
must  discover  how  to  present  information  to  pilots  and  navigators  in  the  most 
easily,  and  rapidly  assimilable  way  possible. 

As  a first  step,  this  study  concentrates  on  ways  in  which  continuous  real- 
time imagery  might  best  be  presented. 
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THE  COCKPIT  ENVIRONMENT 


The  cockpit  environment  is  far  from  ideal. 

Space  is  limited.  One  of  the  prime  reasons  for  the  confusion  of  instru- 
ments that  confront  the  operator  is  that  they  have  to  be  crammed  into  a 
very  restricted  area.  As  discussed  in  the  previous  section,  multisensor 
displays  may  be  of  considerable  importance  in  reducing  this  visual  clutter. 

There  are  other  impediments  in  the  visual  environment.  The  operator's 
view  of  the  world  outside  the  cockpit  is  determined  by  the  design  of  the  air- 
craft; by  the  size  of  the  windows,  and  the  extent  to  which  the  body  of  the 
vehicle  obscures  the  ground.  In  addition,  many  aircraft  are  equipped  with 
some  kind  of  Head  Up  Display  (HUD).  HUDs  are  used  to  present  information 
directly  on  the  windshield.  A more  sophisticated  variant  of  this  is  the 
Helmet  Mounted  Display  (HMD).  In  this  case,  a display  is  mounted  on  the 
visor  of  the  operator's  helmet.  Both  HUDs  and  HMDs  are  useful  visual 
aids.  However,  when  the  observer  is  looking  through  them  to  observe  cock- 
pit instruments,  they  may  be  a hindrance. 

The  space  limitations  also  affect  the  operator  more  directly.  He /she  has 
virtually  no  freedom  of  movement.  On  the  other  hand,  he/she  is  subject  to 
many  forces  caused  by  various  movements,  both  large  (forward  motion,  roll, 
pitch  and  yaw)  and  small  (vibrations). 


We  must  be  aware  of  these  factors,  when  developing  multisensor  displays. 
Anything  that  is  added  to  the  cockpit  environment,  must  be  compact,  and 
robust,  and  should  reduce  visual  clutter. 
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POTENTIAL  ADVANTAGES  OF  MULTISENSOR  DISPLAYS 

Multisensor  displays  could  affect  performance  of  a number  of  the  operator's 
tasks:  reconnaissance,  navigation,  piloting, target  acquisition,  tracking,  fire 
control  and  battle  damage  assessment. 

To  perform  these  tasks,  it  may  be  necessary  for  the  operator  to  use  infor- 
mation from  a number  of  sources,  simultaneously,  and/or  sequentially.  For 
example,  consider  a mission  in  which  the  operator  has  the  task  of  assessing 
battle  damage  on  a particular  installation.  He /she  may  need  to  look  at  map 
information  and  radar  imagery.  He/she  will  then  be  able  to  identify  a particular 
area.  Next,  it  may  be  necessary  to  switch  to  a low  light  level  television 
(LLLTV)  to  see  the  area  in  detail.  Finally,  it  may  be  advisable  to  make  the 
damage  assessment  using  a thermal  image  sensor,  as  well  as  the  television. 


Clearly,  this  task  can  be  performed  if  the  operator  has  a map  and  the  neces- 
sary sensor-display  systems.  However,  it  may  be  easier  to  carry  out  the 
task  using  a multisensor  display.  First,  the  operator  would  look  at  projected 
map  imagery  and  at  the  output  of  the  radar.  With  their  scales  adjusted  appro- 
priately, simultaneous  presentation  on  a single  screen  would  allow  a direct 
comparison.  Having  identified  the  ground  area  of  interest,  the  operator  could 
introduce  the  television  image  by  insetting  it  into  the  appropriate  location,  in 
the  radar  image.  Then,  it  could  be  enlarged  until  it  filled  the  display.  When 
required,  the  thermal  image  of  the  installation  being  inspected  could  be  added. 
Again,  an  inset  might  be  used,  but,  perhaps,  this  time  it  could  be  placed  in  a 
corner  of  the  television  image,  with  a marker  being  used  to  indicate  the  cor- 
responding area  on  the  television  image.  In  this  way,  the  battle  damage 
assessment  could  be  performed,  using  information  obtained  from  the  simul- 
taneous display  of  the  thermal  and  television  images. 
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The  use  of  a single  display  surface  could  lead  to  improvements  in  several 
ways.  The  operator  might  accomplish  the  task  more  rapidly,  or  more 
accurately.  Alternatively,  it  might  require  less  effort,  so  that  the  operator 
has  more  time  or  energy  to  spend  on  other  tasks  that  have  to  be  performed 
at  the  same  time.  Research  is  needed  to  discover  just  which  of  the  various 
image -combining  techniques  will  help  the  operator  improve  his/her  perfor- 
mance in  each  of  the  specific  tasks. 


We  can  expect  new  cockpit  guidelines  to  emerge  from  such  research.  How- 
ever, we  should  not  expect  them  to  emerge  fast,  nor  with  particular  clarity 
at  first.  We  are  dealing  with  complex  techniques  and  complex  behaviors.  It 
will  not  be  easy  to  establish  definitively  the  best  way  of  combining  images. 
For  some  tasks,  one  method  may  be  optimal.  But,  this  method  may  actually 
hinder  the  performance  on  other  tasks. 


SECTION  HI 

MULTI-IMAGE  PRESENTATION 
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There  are  many  ways  of  presenting  a number  of  images.  All  of  them  have 
been  explored  thoroughly  this  century,  first  by  photographers,  then,  by  film 
makers,  and  most  recently,  by  television  directors  and  editors.  Anyone 
who  wants  to  observe  the  full  range  of  possibilities  need  only  sit  in  an  armchair 
and  switch  on  his/her  television.  All  of  the  techniques;  juxtaposition,  super- 
imposition, split  screen,  insetting,  and  variations  on  them,  are  used  either 
in  the  scheduled  programs  or  in  the  commercials  that  interupt  them. 

Occasionally,  they  are  used  to  call  attention  to  themselves.  However,  more 
often,  they  are  used  unobtrusively,  and  to  become  aware  of  them  one  has  to 
pay  close  attention.  The  more  common  uses  of  multiple  images  are  described 
below . 

Two  unbalanced,  individual  images  may  be  combined  to  produce  an  aesthetically 
pleasing,  composite  image.  For  example,  an  image  showing  the  full  figure  of 
a singer  at  the  extreme  right  and  nothing  else  in  the  frame,  may  be  balanced 
by  a second  image,  containing  a close-up  of  his/her  face,  on  the  left,  and 
looking  to  the  right  (Figure  3-1).  Here,  the  two  images  are  combined  to 
produce  a single  aesthetic  entity. 

Two,  or  more,  images  may  be  shown  simultaneously  by  means  of  a split 
screen,  or  insetting,  to  reveal  parallel  action.  For  example,  a long  shot, 
of  the  progress  of  the  ba’l  and  action  around  it  in  a football  game,  may  have 
an  inset,  in  a corner,  of  a static  close-up  of  a player  injured  early  in  the  play 
(Figure  3-2). 
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Figure  3-2.  Composite  of  Two  Parallel  Activities 


Most  other  uses  of  multiple  images  occur  when  the  television  editor,  or 
director  wants  to  make  a transition  from  one  image  to  another.  The  type 
of  transition  used  depends  on  the  action  and  mood  of  the  particular  program. 

These  uses  of  multi-image  presentations  are  of  great  interest  here.  However, 
the  decisions  dictating  their  use  are  quite  different:  they  are  essentially  used 
aesthetically  for  pictorial,  dramatic,  or  atmospheric  effects.  For  us,  the 
emphasis  is  different.  We  are  interested  in  direct  responses  to  images.  We 
are  concerned  with  clarity  of  information  and  speed  of  assimilation. 

In  considering  aircraft  applications  of  multisensor  displays,  two  kinds  of 
problems  must  be  addressed.  The  first  deals  with  the  relative  effectiveness 
of  the  various  ways  of  combining  images,  the  second  with  the  most  efficient 
methods  of  implementing  those  combinations.  The  first  of  these  areas  is 
discussed  in  this  section,  while  the  second  is  dealt  with  in  Section  IV. 

Before  considering  the  potential  effects  of  various  image  combination  method^ 
it  is  necessary  to  mention,  briefly,  the  source  from  which  the  images  may 
emanate.  It  is  also  necessary  to  discuss,  in  rather  more  detail,  the  content 
of  those  images. 

IMAGE  SOURCE 

Cockpit  information  is  provided  by  a number  of  sources.  As  a result,  we 
must  consider  several  different  kinds  of  imagery. 

Low-Light-Level  Television  (LLLTV) 

Low -light-level  television  systems  are  used  mainly  to  provide  ground  infor- 
mation. The  LLLTV  camera  is  generally  slewable,  and  is  controlled  either 
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by  a joystick,  or  by  using  a Helmet  Mounted  Sight  (HMS).  For  most  purposes, 
the  resolution  of  LLLTV  systems  is  equivalent  to  600-800  TV  lines. 


Forward-Looking  Infrared  (FLIR) 

Forward-looking  infrared  systems  also  provide  ground  information  and  are 
slewable.  They  can  be  used  in  two  ways:  first,  as  an  alternative  to  LLLTV 
in  showing  terrain  details;  and  second,  with  low-level  temperature  differences, 
to  highlight  potential  targets. 


Radar 

Radar  is  typically  used  at  a greater  range  than  LLLTV  or  FLIR.  The  radar 
image  has  a geometry  that  is  different  than  that  obtained  with  LLLTV  or 
FLIR.  In  addition,  its  images  take  considerably  longer  to  form. 

Electronic  Dot  Patterns/Stroke  Writing 

Flight  data  can  be  provided  by  the  generation  of  appropriate  symbols. 
Electronic  dot  patterns  are  used  to  convey  alphanumeric  information,  while 
graphical  data  is  given  by  stroke  writing. 

Film 


Film  can  be  used  to  give  moving  map  imagery. 
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IMAGE  CONTENT 


It  may  be  advantageous  for  a pilot  or  navigator  to  receive  information  from 
two,  or  more  image  sources.  The  information  should  be  presented  as  clearly 
and  concisely  as  possible.  It  is  important  to  consider  both  the  task  that  faces  the 
operator,  and  the  content  of  the  images. 

In  particular,  we  must  consider  the  relationship  between  the  content  of  one 
image  and  another.  There  can  be  a wide  range  of  relationships,  from  two 
quite  unrelated  images,  to  two  corresponding  images.  The  relationship  is 
determined  by  the  pointing  direction  of  the  sensors  delivering  the  images,  by 
the  image  geometries,  and  by  the  fields  of  view/magnifications  being  used. 

For  corresponding  images,  two  geometrically  compatible  sensors  (LLLTV 
and  FLIP!  or  radar  with  appropriate  Cartesian  coordinate  conversion)  must 
be  pointing  at  the  same  area  on  the  ground,  with  identical  fields  of  view. 

There  would  be  less  correspondence  if  one  sensor  had  a narrow;  and  the  other 
a wide-angle  field  of  view;  and  still  less,  if  the  two  sensors  were  not  pointing 
in  exactly  the  same  direction,  but  did  cover  some  common  details.  A third 
example  of  a partial  relationship  would  be  provided  by  combining  a FLIR  sensor, 
or  LLLTV  camera,  and  a radar  directed  towards  the  same  general  area. 

As  noted  in  one  of  the  above  examples,  field  of  view  is  variable.  Objects 
within  it  can  be  enlarged,  and  then  inspected  in  more  detail,  at  the  cost  of 
reducing  the  area  covered  by  the  sensor.  With  LLLTV,  FLIR  and  radar, 
field  of  view/ magnification  can  be  changed  discretely  by  a turret  or  step 
zoom  lens.  A continuous  zoom  lens  can  be  used  with  LLLTV  to  provide  a 
continuous  change  in  field  of  view.  This  kind  of  transition  cannot  be  made 
with  current  FLIR  and  radar  systems,  though  for  FLIR  at  least,  it  may  be 
desirable. 
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A continuous  zoom  lens  allows  two  options  not  possible  with  a turret  or  step 
zoom  lens.  First,  smooth,  fluid  transitions  can  be  made  with  the  detail  of 
interest  remaining  in  view  and  becoming  progressively  larger.  Second,  the 
zoom  can  be  stopped  at  any  point  between  the  maximum  and  minimum  fields 
of  view.  With  the  other  lenses,  only  two  or  three  fixed  fields  of  view  are 
available,  and.  during  a change  from  one  view  to  another,  the  observer' s 
screen  goes  blank. 

It  would  be  prohibitively  costly  to  produce  a continuous  zoom  lens  for  thermal 
image  systems.  However,  the  first  of  the  two  continuous  zoom  options  could 
be  provided  by  adding  an  electronic  storage  unit  and  making  use  of  the  existing 
step  zoom  lens  and  display.  A wide-angle  field-of-view  image  would  be 
stored  on  the  storage  unit.  Then,  while  the  lens  was  being  switched  to  a 
narrow  field  of  view,  there  would  be  a continuous,  electronic  zoom-in  on  the 
stored  image.  Honeywell  hopes  to  investigate  this  fake,  continuous  zoom 
effect  in  the  near  future. 

Changes  in  field  of  view,  whether  mediated  by  step  zoom,  continuous  zoom 
or  fake  continuous  zoom,  have  a profound  effect  on  image  content.  Changes 
in  pointing  angle  and  image  geometry  have  similar,  drastic  effects.  A con- 
sideration of  image  content  is  of  considerable  importance  when  one  is  attempt- 
ing to  present  to  an  observer  more  than  one  image  at  a time. 

USES  OF  MULTI-IMAGE  PRESENTATION 

Imagery  from  a number  of  different  sources  can  be  used  in  several  ways. 

First,  an  observer  may  need  to  compare  the  outputs  of  two,  or  more  sources 
to  derive  a kind  of  information  composite.  An  example  was  given  under 
"Potential  advantages  of  multisensor  displays."  There  it  was  suggested  that 
an  observer  might  compare  information  gleaned  from  a FLIR  inset,  and  from 
the  marked  area  of  a full-screen  LLLTV  image. 
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This  use  is  related  to  the  first  example  of  multi-image  usage  in  commercial 
television.  Here,  an  information  composite  is  derived  from  two  or  more 
sources.  There,  one  unbalanced  image  was  superimposed  on  another  to 
produce  an  aesthetically  pleasing  composite  image. 

If  our  first  use  is  related  to  this  example,  our  second  is  directly  equivalent 
to  it.  In  some  cases,  it  is  possible  to  produce  an  enhanced,  composite  image 
by  combining  separate  pieces  of  imagery  before  showing  them  on  the  observer' s 
screen.  Our  objective  is  not  an  aesthetic  one,  but  rather,  relates  to  the 
clarity  of  the  resultant  information. 

Thirdly,  the  observer  may  need  to  monitor  infor  ,’ion  coming  from  more 
than  one  source.  Each  image  conveying  information  should  be  distinct.  Our 
second  commercial  television  example,  of  parallel  action  in  a football  game, 
is  relevant  in  this  case.  There,  parallel  action  was  indicated  by  insetting, 
though  in  fact  commercial  television  makes  use  of  other  methods,  including 
split  screen,  juxtaposition  in  time,  and  (less  frequently)  super-imposition. 

We  must  consider  these  alternative  methods  of  monitoring  parallel  informa- 
tion. 

Fourthly,  transitions  from  one  image  source  to  another  may  be  facilitated  by 
presenting  both  images  simultaneously,  around  the  time  of  the  transition. 

This  is  similar  to  the  third  example  from  commercial  television,  although 
our  objectives  in  making  transitions  may  be  rather  different.  We  are  not 
interested  in  aesthetics,  drama, or  mood;  but  in  clarity  and  ease  of  under- 
standing. As  a result,  a number  of  editing  techniques  will  be  of  no  use.  We 
will  not  want  to  fade-out  from  our  first  image  to  darkness,  and  then  fade-in 
the  second  one,  although  we  may  consider  dissolving  or  mixing  the  two  images 
by  means  of  superimposition.  Similarly,  we  will  not  be  interested  in  irising- 
out  from  image  to  darkness  and  then  irising-in  to  the  new  image;  though 
again,  we  may  think  of  insetting  the  second  image  in  the  first,  and  then 
gradually  enlarging  the  inset.  Nor  will  we  want  a turn-over,  where  the 
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first  image  appears  to  turn  over  to  reveal  the  second  on  its  back.  In  this 
latter  case,  both  images  are  distorted  during  the  transition.  However,  we 
may  be  interested  in  using  either  a split  screen  during  the  image  change  or 
a wipe,  in  which  the  new  image  is  revealed  by  a line  that  wipes  off  the  pre- 
vious image. 

Some  form  of  multiple -image  presentation  may  be  effective  for  each  of  the 
uses  identified  above.  However,  different  forms  of  presentation  may  be 
needed  to  produce  the  most  comprehensive  information  composites  and  the 
clearest  composite  images  to  convey  parallel  information  succinctly  and 
to  give  smoother  transitions.  We  now  need  to  consider  the  various  methods 
of  presentation  available. 


METHODS  OF  PRESENTING  MULTIPLE  IMAGES 

There  are  several  ways  of  presenting  multiple  images: 

1.  Images  filling  the  whole  screen  may  be  shown  simultaneously 
on  multiple  screens. 

2.  Images  filling  the  whole  screen  may  be  shown  simultaneously, 
superimposed  upon  each  other  on  a single  screen. 

3.  Images  filling  the  whole  screen  may  be  shown  separately, 
juxtaposed  in  time,  on  a single  screen. 

4.  Images  occupying  equal  proportions  of  the  display  space  may 
be  shown  simultaneously  on  a single  screen.  This  is  known  as 
a split-screen  presentation. 

5.  A full-screen  image  may  be  shown  with  a designated  portion 
of  it  overlayed  by  the  superimposed  inset  of  a second  image. 
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6.  A full-screen  image  may  be  shown  with  an  inset  of  a 
second  image  replacing  a designated  portion  of  it. 


7.  A variant  on  Number  6,  with  the  inset  image  placed 

in  one  corner  of  the  main  image,  and  a marker  indicating 
an  area  of  special  interest  in  the  latter. 

8.  A variant  on  Number  7,  with  an  inset  again  placed  in  a 
corner  of  the  main  image.  In  this  case,  there  is  no 
marker,  as  there  is  no  special  relationship  between  the  two 
images. 

MATCHING  METHODS  TO  USES 

We  have  described  eight  different  methods  of  presenting  multiple  images, 
and  four  main  uses  of  such  a presentation.  The  next  step,  is  to  discuss 
which  method,  or  methods,  might  be  most  appropriate  for  each  use.  Table 
3-1  lists  the  various  methods  and  uses.  An  indication  is  given  of  those 
methods  that  are  inappropriate  for  some  uses,  those  that  may  be  useful, 
and  those  we  judge  to  be  most  promising. 

It  should  be  noted  that  Table  3-1  presents  judgments.  An  experimental 
investigation  is  required  before  any  definitive  matches  of  methods  and  uses 
can  be  stated. 

The  table  is  discussed  in  terms  of  the  four  identified  uses  of  multiple-image 
presentation. 
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Table  3-1.  Applicability  of  Eight  Methods  of  Presenting 

Multiple  Images  to  Each  of  Four  Ways  of  Using 
Those  Images.  (Key:  X = Inappropriate; 

P = Possible;  M = Most  Promising) 


U ses 

Methods 

Information 

Composite 

Composite 

Image 

Monitoring 

Parallel 

Information 

T ransitions 

1 . Multiple  Screens 

P 

X 

P 

P 

2 . Full  Image 

Superimposition 

P 

M 

P 

P 

3.  Juxtaposition  in  Time 

M 

P 

M 

M 

4.  Split  Screen 

M 

X 

M 

P 

5.  Superimposed  Inset 

P 

M 

X 

X 

6.  Inset  (Replacement) 

X 

M 

X 

M 

7.  Inset  with  Marker 

M 

X 

X 

P 

8.  Inset,  No  Relationship 

X 

X 

M 

P 

Information  Composite 

In  this  use,  the  observer  has  to  construct  a composite  of  the  information 
he/ she  extracts  from  two  or  more  distinct  image  sources. 

All  three  methods  using  full-screen  images  could  be  used.  With  Method  1, 
multiple  screens,  the  distance  between  the  displays  showing  the  imagery 
should  be  the  minimum.  The  farther  apart  the  relevant  displays  happen  to 
be,  the  larger  the  eye  movements  needed  to  look  from  one  to  the  other,  and 
the  less  satisfactory  this  method  is  likely  to  be. 
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Method  2,  involving  superimposed  full  screen  images,  could  be  used.  However, 
some  coding  of  the  images,  perhaps  by  color,  would  be  necessary  if  they 
were  to  be  easily  distinguishable  from  each  other. 

The  third  full-screen  method  is  more  promising.  Observers  may  find  they 
are  able  to  combine  information  judiciously,  when  it  is  derived  from  images 
that  are  shown  alternately  on  a single  screen.  Several  questions  need  to  be 
investigated:  should  the  observer  control  the  alternation  process?  Is  there 
an  optimal  alternation  rate?  Does  optimal  rate  vary  with  image  content? 

Method  4,  split-screen  presentation,  also  may  be  appropriate  for  this  usage. 

It  is  similar  to  multiple-screen  presentation,  but  has  at  least  one  advantage. 

Since  the  images  are  presented  side  by  side  on  a single  display,  inter-image 
distance  is  at  a minimum.  This  method  will  be  suitable,  if  there  is  sufficient 
resolution  on  the  half-screen  images  to  see  the  details  of  interest  clearly. 

Two  of  the  four  methods  involving  insets  are  inappropriate  for  this  usage. 

Method  G,  in  which  the  area  of  interest  in  the  full  image  is  replaced  by  an 
inset,  is  clearly  unsuitable,  since  no  comparison  would  be  possible.  And,  in 
Method  8,  since  there  is  no  special  relationship  between  the  inserted  image 
and  the  full  image,  no  comparison  could  be  made. 

Method  5,  in  which  an  inset  is  superimposed,  might  be  useful.  As  with  full- 
image  superimposition,  coding  would  be  necessary,  or  the  inset  may  simply 
merge  into  the  full  image. 

The  last  inset, Method  7,  joins  Methods  3 (juxtaposition  in  time)  and  4 (split 
screen),  in  having  the  most  potential  for  this  use.  In  this  case,  the  area  of 
interest  on  the  full  screen  is  marked,  and  an  inset  shows  the  area  via  another 
sensor  system.  Resolution  is  less  likely  to  be  a problem  than  in  Ihe  split 
screen  method  since  the  area  of  interest  can  be  shown  with  a maximum  resolution 
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on  both  inset  and  full  screen.  In  addition,  unlike  either  the  juxtaposition  in 
time  or  split  screen  methods,  the  details  for  comparison  can  be  pinpointed 
more  accurately. 

Composite  Images 

In  this  case,  the  observer  receives  a single  image  that  is  a composite  of  two, 
or  more,  combined  images. 

Methods  1 (multiple  screens),  4 (split  screen),  7 and  8 (both  with  insets  in 
the  corner  of  a full  image)  are  all  clearly  inappropriate. 

Method  3,  juxtaposition  in  time,  is  a possibility,  but  only  because  of  a 
technicality.  If  the  alternation  rate  of  two  distinct  images  is  sufficiently 
rapid,  the  two  images  appear  to  be  superimposed  upon  one  another.  If  a 
suitable  rate  were  used,  a composite  image  would  be  seen.  However,  if 
alternation  were  used  in  this  way,  it  should  really  be  considered  as  an 
alternative  way  of  implementing  superimposition  Method  2. 

The  two  superimposition  methods,  with  full  images  (Method  2)  and  with  an 
inset  (Method  5),  are  clearly  of  great  interest  here.  However,  there  may  be 
problems  in  balancing  the  images  that  are  to  be  combined.  First,  misalignment 
of  the  images  could  lead  to  interference  that  would  degrade  each  image. 

Second,  the  gray  scale,  or  dynamic  range,  of  the  individual  images  should  be 
matched,  otherwise  the  display  with  the  larger  dynamic  range  would  dominate 
the  other(s).  Third,  the  signal  to  noise  ratio  in  the  images  needs  to  be  balanced, 
since  a high  noise  content  in  one  image  would  degrade  information  from  another. 
These  technical  problems  need  to  be  dealt  with  if  this  form  of  image  enhance- 
ment is  to  be  conducted  in  real  time. 
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Spotlighting  is  a special  case  of  superimposition  Method  2.  Here,  the  low 
temperature  differences  picked  up  by  the  FLIR  sensor  would  be  supressed. 
Then,  the  high  temperature  differences  could  be  used  to  indicate  potential 
targets.  The  FLIR  image  might  be  defocussed  slightly,  so  that  it  did  not 
interfere  with  the  detail  information  contained  in  a LLLTV  image.  The  latter 
would  be  used  to  identify  targets  from  amongst  the  candidates  suggested  by 
the  FLIR. 

One  last  method  needs  to  be  discussed  in  relation  to  this  usage:  Method  6, 
involving  an  inset  that  replaces  a portion  of  the  full  image.  In  this  case,  the 
observer  may  need  to  combine  terrain  background  information,  perhaps  from 
a LLLTV,  with  a small  inset  of  FLIR  target  information.  The  FLIR  sensor 
might  be  used  by  the  observer  to  search  around  the  LLLTV  image.  There 
would  be  no  alignment  problem,  nor  would  it  be  necessary  for  both  sensors 
to  have  the  same  scale. 

Monitoring  Parallel  Information 

In  this  use  of  multiple  image  presentation,  the  observer's  task  is  to  monitor 
information  coming  from  two  or  more  sources  continuously.  Each  image 
needs  to  be  distinct.  There  are  similarities  between  this  case  and  the  first 
usage  involving  an  information  composite.  In  both  cases  it  is  necessarv  for 
the  images  to  remain  clearly  distinguishable  from  each  other. 

As  before,  the  three  full-screen  methods  could  be  used.  With  a multiple- 
screen  presentation,  the  intei -display  distance  should  be  as  small  as  possible. 
With  superimposition,  coding  would  once  more  be  needed  to  ensure  that  the 
images  are  distinct.  And  again,  alternation  in  time  is  the  more  promising 
method,  with  a similar  investigation  into  alternation  control  and  rate  being 
called  for.  Method  4,  involving  split-screen  presentation,  would  again  be 
an  appropriate  method. 
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There  are  differences  between  the  two  usages  that  emerge  when  we  consider 
the  various  insetting  methods.  Those  methods  (5,  6 and  7)  that  deal  with 
ways  in  which  the  inset  image  is  related  to  the  full-screen  image  are  all 
inappropriate.  Only  Method  8,  in  which  an  unrelated  image  is  inset  in  a 
corner  of  the  main  image,  is  revealed  as  a candidate. 

Transitions 


In  making  transitions  from  one  image  to  another,  a number  of  multiple  image 
presentations  may  be  useful.  In  fact,  of  the  eight  methods  that  were  identified, 
only  one  can  be  eliminated.  There  seems  to  be  little  point  in  using  Method  5. 
The  combination  of  superimposition  and  insetting  is  unlikely  to  ameliorate 
transitions,  although  either  alone  could  be  of  assistance. 

With  multiple  screens,  transitions  would  be  mediated  by  eye  movements  from 
one  sensor  display  to  another.  This  method  seems  unlikely  to  be  the  best, 
although  it  may  be  as  good  as  Methods  2 (superimposition),  4 (split  screen), 
and  7 (inset  placed  in  one  corner,  with  marker  in  the  area  of  intei’est),  and 
it  is  probably  better  than  Method  8 (where  an  unrelated  image  is  inset  in  a 
corner  of  the  full  screen  image).  It  may  also  be  as  useful  as  one  commercial 
television  editing  technique  that  was  not  covered  by  our  listing  of  methods  of 
presenting  multiple  images.  This  technique  is  the  wipe.  In  this,  one  image 
is  swept  away  by  the  leading  edge  of  the  second  image. 

The  two  most  promising  transitional  techniques  are  Methods  3 and  6.  The 
former,  juxtaposition  in  time,  here  becomes  direct-cutting.  The  latter, 
with  an  inset  replacing  a specific  area  of  interest,  may  be  particularly 
useful  in  target  acquisition  operations.  After  the  image  was  inset,  the  trans- 
ition would  be  effected  by  allowing  the  inset  to  grow  gradually  until  it  filled 
the  whole  scene,  and  had  completely  replaced  the  initial  image.  There  are 


two  ways  that  the  inset  image  could  be  expanded.  Either  part  of  the  second 
image  could  be  inset,  and  the  rest  revealled  gradually.  Or  the  whole  of  the 
second  image  could  be  minified,  inset,  and  then  magnified  and  centered  on 
the  display.  The  latter  is  rarely  seen  and  harder  to  implement,  but  may  be 
interesting  for  our  purposes. 

While  discussing  transitions,  changes  of  field  of  view  involving  a single 
sensor  should  be  mentioned.  When  a turret,  or  step  zoom  lens  is  used, 
the  observer' s display  goes  blank  during  the  transition  from  one  field  of 
view  to  another.  This  may  result  in  some  disorientation  or  confusion.  It 
may  be  difficult  for  the  observer  to  decide  exactly  what  is  in  the  narrow 
field  of  view,  and/or  how  it  relates  to  the  previously  visible,  wider  angle 
view.  These  potentially  disconcerting  effects  may  delay  the  observer' s 
response.  The  delay  may  only  be  short.  Nevertheless,  for  tasks  such  as 
target  acquisition,  it  may  be  of  considerable  importance. 

It  is  possible  that  the  use  of  a real,  or  fake  continuous  zoom  lens  system 
would  avoid  such  effects  and  lead  to  a more  efficient  performance. 

RECOMMENDATIONS 

There  are  32  combinations  of  presentation  methods  and  presentation  uses  listed 
in  Table  3-1.  As  the  table  and  the  subsequent  discussion  show,  1 0 of  these 
combinations  can  be  eliminated  from  further  consideration.  However,  22 
combinations  seem  to  have  some  possibilities.  Of  these,  11  seem  particularly 
promising.  With  the  exception  of  the  multi-screen  method,  all  seven  remain- 
ing methods,  i.e.  all  methods  involving  a single  screen,  seem  to  be  promising 
in  at  least  one  usage.  Because  of  this,  we  will  discuss  ways  of  implementing 
each  of  these  methods  in  the  next  section. 
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SECTION  IV 
IMPLEMENTATION 


INTRODUCTION 

Any  airborne  multifunction  display  must  be  able  to  present  information  from 
a variety  of  sensors  and  data  sources.  The  multifunction  display  must  not 
only  be  compatible  with  various  sensors  but  also  with  computer-generated 
symbology  der  d from  instrumentation  sensors  and/or  weapon  systems. 

The  input  sources  which  appear  to  require  the  highest  performance  from  an 
airborne  display  are  those  which  form  images.  The  inherent  flexibility  of 
electronic  displays  such  as  the  cathode  ray  tube  is  particularly  attractive  for 
multifunction  use.  While  the  inherent  flexibility  permits  a high  degree  of 
optimization  to  a specific  system  under  specific  viewing  conditions,  obviously 
a multifunction  display  has  a large  range  of  performance  requirements  and 
presents  a formidable  task  to  the  display  designer. 

A representative  list  of  sensors  and  other  data  sources  for  a multifunction 
display  on  a military  aircraft  includes  the  following: 

I.ow  light  level  TV  (LLLTV) 

Forward  looking  infrared  (FLIR) 

Radar 

Computer  generated  and  processed  information  (e.g.  , 
weapon  aiming,  flight  information,  and  aircraft  systems) 

Dedicated  electronic  cockpit  displays  have  been  in  use  for  many  years.  But 
with  current  proliferation  of  sensor  systems  and  increased  quantity  of  pro- 
cessed data  available  from  digital  computing  systems,  a need  exists  to  mini- 
mize crew  workload  within  instrument  panel  space.  The  optimal  approach 
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to  design  of  an  integrated,  multisensor,  multifunction  display  system  seems 
to  be  a raster  based  integrated  display  system.  In  it.  all  sensor  data  are 
converted  to  the  standard  video  line  rate  prior  to  mixing  with  any  synthetically 
generated  symbology.  Various  approaches  to  combining  sensor  imagery  with 
synthetic  symbology  will  be  discussed  in  this  section. 

It  is  unlikely  that  the  resolution  and  gray  scale  requirements  for  radar  infor- 
mation will  exceed  that  of  the  optical  image  forming  sensors;  hence,  it  wi  1 
not  represent  a limiting  condition  on  display  performance.  It  is  unlikely  that 
symbology  for  target  designation  or  navigational  data  for  flight  information 
would  define  the  limiting  conditions  for  a multifunction  display.  Rather,  it 
would  need  to  be  compatible  in  readability,  contrast,  accuracy  and  resolution 
with  associated  imaging  sensor  information. 

The  radar  sensor  is  basically  incompatibility  with  a multifunction  (TV  raster) 
display  due  to  its  relatively  slow  scan  rates.  A radar  sensor  requires  either 
a display  with  a long  persistence  or  some  sort  of  scan  conversion  to  display 
the  imagery  at  TV  refresh  rates. 

Two  types  of  synthetic  symbol  generation  are  stroke  writing  and  raster  writing. 
Stroke  writing  is  produced  by  applying  analog  x and  y signals  to  the  x and  y 
deflection  circuits  of  the  display  device,  causing  the  spot  to  deflect  on  the 
CRT  face  correspondingly.  The  spot  is  switched  OFF  while  it  is  being  moved 
into  position  and  ON  while  it  is  being  moved  to  create  a line  or  symbol. 

Raster  writing  (as  in  television)  is  produced  by  generating  x and  y triangular 
waveforms  locked  to  the  synchronizing  signals  from  the  source  in  the  x and  y 
deflection  circuit  of  the  display  device.  Triangular  waveforms  cause  the 
spot  to  trace  out  a large  number  of  horizontal  lines.  The  brightness  of  the 
spot  is  varied  to  generate  the  symbol  as  horizontal  lines  comprising  it  reach 
the  desired  location  on  the  screen. 
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A display  designed  originally  for  raster  writing  cannot  easily  be  modified  to 
generate  stroke  written  symbols.  However,  a stroke  writing  display  can 
usually  generate  raster  writing  without  modification. 


The  TV  raster  approach  has  the  advantage  of  being  cost  effective  with  low 
power  requirements.  However,  the  stroke  writing  display  has  the  capability 
of  generating  high  resolution  symbols  between  fields  of  the  standard  TV  dis- 
play format.  When  this  is  done  the  symbol  brightness  adds  to  the  brightness 
of  the  raster  imagery,  and  the  synthetic  data  are  always  legible.  In  other 
words,  the  symbol  becomes  whiter  than  the  wtute  of  the  image  in  the  raster. 
Raster  type  displays  must  use  some  techniques  such  as  black  bordering  for 
the  synthetic  symbol  to  be  visible  against  peak  white  portions  of  the  scene. 
However,  this  obliterates  some  of  the  imagery.  Conflicting  results  of 
various  optional  design  approaches  must  be  traded  off  against  requirements 
of  the  specific  aircraft’s  mission. 

A detailed  analysis  of  proposed  image  combining  techniques  has  not  been 
undertaken  in  this  study.  A meaningful  analysis  of  MTF,  resolution, 
brightness,  contrast,  visual  acuity,  probability  of  detection,  recognition, 
and  identification  requires  the  definition  of  a specific  sensor /combiner/ 
display  system  for  a specific  application.  Therefore,  it  appears  that  the 
most  cost-effective  approach  to  the  multifunction  display  problem  is  to 
select  a candidate  from  the  alternatives  in  this  study  on  a qualitative  basis. 

A follow-on  effort  should  then  be  implemented  to  perform  a quantitative 
analysis  of  its  suitability  to  its  specific  application  and  aircraft. 
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The  main  thrust  of  this  study  has  been  directed  toward  the  standard  television 
monitor.  Advantages  of  this  display  type  are  high  resolution  and  high  bright- 
ness when  combined  with  a narrow  band  phosphor  and  matched  spectral 
filter  for  visibility  under  high  ambient  light  conditions.  The  television  moni- 
tor's tuned  CRT  display  has  the  advantages  of  simplicity,  reliability,  main- 
tainability, lower  power  and  low  cost  as  compared  to  other  types  of  displays. 
Before  adopting  it,  however,  a very  important  tradeoff  in  display  design  is 
required  between  the  number  of  symbols  and  the  ability  to  write  them  during 
the  field  fly-back  time  of  the  raster  display.  This  tradeoff  is  among  the 
display  brightness,  the  writing  speed,  and  the  number  of  symbols  to  be 
imaged. 


In  addition  to  raster  and  stroke -written  CRTs,  other  display  materials  will 
be  discussed.  These  include  plasma  panels,  light  emitting  diode  arrays, 
electro-luminescent  panels  and  liquid  crystal  panels.  This  study  is  primarily 
concerned  with  the  heads  down  display  rather  than  the  heads  up  or  helmet- 
mounted  display,  though  some  image  combining  techniques  discussed  here 
also  apply  to  them. 


SYSTEM  REQUIREMENTS 

Table  4-1  summarizes  the  requirements  on  which  the  analysis  of  various 
image  combining  techniques  is  based.  These  requirements  do  not  relate  to 
a specific  application  but  have  been  selected  to  reflect  a generic  worst  case 
for  a multifunction  display.  We  assume  the  sensors  have  high  quality  imagery 
with  EO  sensors  having  a TV  compatible  video  output.  Five  cases  for  image 
combinations  have  been  selected.  We  assume  also  that  the  radar  and  moving 
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Table  4-1.  Preliminary  Requirements  forCandidate 
Image  Combining  OonceDts 


• High-quality  radar  with  forward  sector  PPI  coverage  at  low  frame  rates  and 
low  number  of  gray  shades. 

• High-quality  forward  oblique  FLIR  and  I I I U hig  tr  rat< 
number  of  gray  shades. 


Annotation 


• Moving  map 

• Symbols  and  graphics 


Image  Combination  Cases 


• Radar  and  moving  nap  sequentially  or  sii 

• M or  inset  radai 

a different  scale  and  geometry. 

• FLIR  imag<  ry  insert  at  different 
vice  versa. 

• Combine  FLIR  and  LLI  TV  i lager > sequentially  01 
scale  and  geometry. 

• Syml  ol  and  gi  notation 

Cockpit  Environment 

• High  ambient  illumination 

• Limited  head  motion  requirement 

• Severe  vibration 

! Display  (Design  Goals) 

• Pilot  - 11 . 4 x 14  x 28  w it  10.  -2  I lines) 

• Navigator  - 21.  6 x IT.  1 x 41.  2 err.  with  1 x 13.4  cm  active  ar  a. 

(875  TV  lines) 

• \ iewing  distance  - 51  to  76  cm. 


• \ir craft  flight  control: 

following,  threat  avoidance . rendezvous.  approach  md  landing. 

• ‘lentatioi  d kpoint  locatioi  ignatio 

and  position  entry  for  navigation  update. 

• Attack  itioi  i 

location,  designation  position  onti  and  tracking. 


• Simpb 


Reliability 


• Lov 


map  Imagery  have  the  same  scale  and  geometry.  No  requirement  has  been 
assumed  to  present  the  radar  and  EO  imagery  at  the  same  scale  and  geometry. 
The  radar  is  assumed  to  have  a forward  looking  PPI  format.  The  EO  sensors 
are  assumed  to  have  a forward  oblique  geometry  but  can  be  at  different  scales, 
geometries  or  TV  line  rates.  Symbol  annotation  is  assumed  for  both  radar 
and  EO  imagery. 

A cockpit  environment  with  a high  ambient  illumination,  limited  head  motion 
and  severe  vibration  (as  in  a helicopter)  is  assumed.  Air  crew  tasks  are 
assumed  to  be  fairly  complex  and  require  high-quality  radar  and  EO  imagery. 
Design  goals  for  the  multifunction  display  are  high  reliability,  simplicity, 
low  cost  and  a minimum  cockpit  space. 


SENSOR.  DISPLAY  AND  IMAGE  COMBINING 
COMPONENT  CHARACTERISTICS 

Sensors 

Radar  --  A forward-looking  PPI  sector,  slow  scan  radar  has  been  selected 
as  one  of  the  imaging  sensors.  A worst  case  for  displaving  information  has 
been  selected  that  assumes  a good  quality  ground  map  image  and  a require- 
ment that  the  operator  recognize  the  target.  These  requirements  impact  the 
type  of  display  that  can  be  used.  A tvpical  forward-looking  radar  requires 
about  one  second  to  form  a complete  image.  It  demands,  therefore,  that  the 
display  system  store  the  image  over  that  time  if  the  air  crew  is  to  see  it  in 
its  entirety.  A television  sensor  takes  only  one- thirtieth  of  a second  to  form 
a complete  image,  and  television  information  will  be  lost  if  it  is  stored  for 
one  second.  Consequently,  some  sort  of  scan  conversion  must  be  used  for 


the  radar  sensor  to  convert  it  to  TV  compatible  video.  Studies  of  operator 
performance  indicate  that  eight  shades  of  grey  ( V^"  ratio)  are  adequate  for 
radar  imagery,  and  at  least  16  are  needed  for  EO  imagery^.  It  is  unlikely 
that  resolution  requirements  for  the  radar  display  will  exceed  the  resolution 
requirements  for  an  EO  sensor.  Hence,  radar  imagery  does  not  represent 
a limiting  condition  for  display  performance^. 

Forward-Looking  Infrared  Sensor  (FLIR)  --  We  assume  that  the  FLIR  sensor, 
whether  serial  scan  or  pushbroom  type,  will  have  output  vLdeo  in  TV  com- 
patible format.  The  field  of  view  of  the  FLIR  sensor  ranges  from  65  for 
navigational  systems  to  about  4°  for  target  identification  and  tracking  systems. 
Current  spatial  resolutions  are  about  0.2  mrad.  which  requires  350  resolution 
elements  across  the  four-degree,  narrow  field  of  view.  Gray  scale  require- 
ments are  about  16  shades  of  gray  for  the  high-quality  imagery  assumed  for 
this  study.  The  ground  footprint  of  the  forward  oblique  field  of  view  of  the 
FLIR  sensor  presents  an  image  with  a varying  ground  resolution  from  the 
bottom  to  the  top  of  the  display.  Therefore  the  FLIR  image  geometry  is  not 
compatible  with  the  cartesian  ground  presentation  of  the  radar  sensor. 

Low  Light  Level  TV  Sensor  (LLLTV)  --  Numerous  LLI.TV  systems  are 
available  from  various  combinations  of  camera  tubes  and  intensif iers.  Under 
reasonable  light  levels,  these  systems  have  a resolution  of  between  600  to  800 
TV  lines  per  picture  height.  Consequently  the  LLLTV  systems  with  a 16 
gray  scale  requirement  represent  the  worst  case  for  display  requirements. 


Symbol  Generator 

An  electronic  display  system  must  be  capable  of  generating  a variety  of  sym- 
bology and  superimposing  this  on  a picture  of  the  outside  world  provided  by 
the  FLIR,  Lid. TV  or  radar.  An  example  of  symbology  for  a system  without 
imagery  is  shown  in  Figure  4-1.  Here  (in  the  A-7  aircraft)  symbology  is 
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HEADING 


Figure  4-1.  A-7  Weapon  Delivery  HUD 


presented  to  show  target  designation,  automatic  release  point  and  the  bomb 
path.  The  display  also  shows  the  air  speed,  pull-up  cue,  radar  altitude, 
horizon  line  and  heading.  Figures  4-2,  4-3  and  4-4  show  F-16  displays 
combining  weapon  delivery  vectors  with  FLIR  or  LLLTV  imagery.  Figure  4-2 
shows  the  weapon  status  for  the  continuously  computed  impact  point  and  bomb 
fall  line  and  velocity  vector.  A dive  toss  mode  is  shown  in  Figure  4-3  against 
a bridge,  and  Figure  4-4  shows  the  gun  or  rocket  steering  line  and  gun  rounds 
remaining  against  a truck  target. 

A typical  symbol  display  system  configuration  is  shown  in  Figure  4-5.  The 
symbol  generation  facility  consists  of  three  main  units--a  computer,  a wave 
form  generator  and  interface  unit.  The  interface  unit  receives  signals  from 
various  aircraft  data  sources  and  directs  them  to  the  computer,  which  calcu- 
lates parameters  of  the  display  symbols.  These  parameters  are  in  turn  fed 
to  the  waveform  generator.  The  FLIR  or  LLLTV  and  Lts  associated  control 
unit  supply  the  video  to  the  waveform  generator,  winch  mixes  it  with  symbology 
to  produce  a composite  video  waveform^.  This  system  is  a raster-based 
electronic  display  that  mixes  the  symbology  with  the  raster  of  the  sensor 
imagery. 

An  alternate  procedure  is  to  replace  the  waveform  generator  with  a display 
processor  that  generates  signals  to  stroke  write  on  the  display.  This  re- 
quires a direct  drive  display  system.  Symbols  in  the  stroke  write  mode 
are  usually  written  in  the  flyback  time  between  fields  of  the  sensor  signal. 

The  number  of  symbols  that  can  be  written  in  the  fixed  flyback  time  is 
directly  proportional  to  the  speed  of  the  deflection  circuits  and  inversely 
proportional  to  the  required  brightness.  The  writing  speed  or  rate  can  only 
be  improved  by  increasing  the  deflection  circuit's  bandwidth  and  power. 

Several  computer/display  command  interfaces  are  possible.  The  easiest 
for  the  display  but  the  most  time  consuming  for  the  computer  is  a point-by- 
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point  method  where  each  picture  element  is  addressed  sequentially  by  the 
computer.  Another  method,  requiring  less  computing  power,  uses  a display 
device  that  can  draw  predetermined  objects  in  specific  orientations  and  the 
computer  only  specifies  that  location.  This  limits  the  flexibility  of  the  display 
significantly.  Somewhere  between  these  two  levels  of  complexity  is  the  line 
orientated  display.  Here  the  computer  specifies  lines  as  vectors  with  initial 
coordinates,  angles,  and  lengths;  the  display  hardware  does  the  calculations 
necessary  to  draw  them.  The  restriction  in  this  method  is  that  ail  pictures 
must  be  line  segments.  Curved  lines  cannot  be  drawn  directly  but  may  be 
approximated  by  many  short  line  segments.  A CRT  is  easily  controlled  by  a 
line-orientated  display  generator.  High-speed  deflection  circuits  are  neces- 
sary in  the  CRT  to  allow  all  required  lines  to  be  drawn  before  the  persistence 
of  the  phosphor  requires  them  to  be  redrawn.  This  method  of  positioning  the 
beam  to  arbitrary  location  is  known  as  a random  scanning  technique. 

The  standard  television  monitor  utilizes  a fixed  raster  scan  technique.  With 
this  method  the  position  of  the  trace  cannot  be  selected  by  the  display  generator. 
Scan  positioning  the  electron  beam  is  a known  function  of  time.  Only  the 
intensity  can  be  controlled  externally.  To  draw  a line,  all  points  through 
which  the  line  passes  must  be  known  and  constantly  compared  with  the  actual 
location  of  the  beam.  When  they  correspond,  the  beam  is  unblanked.  This 
method  works  well  when  only  horizontal  or  vertical  lines  need  to  be  drawn. 
However,  the  appearance  of  diagonal  or  curved  lines  is  uneven  or  ragged  due 
to  the  step-wise  construction.  The  random  scan  displays  are  preferred  when 
short  curved  or  diagonal  lines  are  required. 

Moving  Map 

Current  moving  map  displays  have  a high  resolution  projected  map  image  ob- 
tained by  using  film  cassettes,  servo  drive  and  special  optics.  The  map 
imagery  is  usually  superimposed  on  CRT-generated  symbols  and  a CRT- 
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generated  navigation  radar  image.  Usefulness  of  the  radar  image  and  map 
display  is  greatly  enhanced  if  they  are  accurately  registered.  One  technique 
for  combining  these  images  uses  a rear  port  CRT  as  shown  n Figure  4-6. 

This  particular  configuration  was  proposed  by  Control  Data  for  the  Boeing 
B-l  system"^.  The  film  is  scanned  by  a rotating  periscope  as  shown  in  Fig- 
ure 4-7.  Rotation  of  the  map  imagery  is  accomplished  by  orientation  rotator 
optics;  examples  are  shown  in  Figure  4-8.  The  rotating  periscope  scan.-  th< 
film  in  the  North/South  direction,  and  the  combination  of  the  rotating  peri- 
scope and  film  motion  scans  the  film  in  the  East/ West  direction.  This  par- 
ticular system  contains  a hundred  feet  of  70- millimeter  film  with  three  mag- 
nification scales,  as  shown  in  Figure  4-9.  The  performance  summary  in 
Figure  4-10  indicates  a limited  brightness  of  500  foot-lamberts,  which  is 
typical  of  a rear  port  CRT  system. 

The  fundamental  problem  in  this  approach  is  that  the  phosphor  must  be  re- 
duced in  thickness  when  a CRT  is  used  as  a back  projected  screen.  The  bright- 
ness level  of  a radar  image  may  then  be  inadequate  under  high  ambient  illu- 
mination conditions. 

The  Control  Data  CDS-1  system  employs  contast  enhancement  techniques  that 
include  a fiber  optics  face  plate,  a neutral  density  filter  and  antireflection 
coatings.  A field  lens  may  be  used  for  extremely  narrow  viewing  lobe  appli- 
cations. Even  with  these  techniques,  visibility  of  the  combined  image  in  the 
presence  of  high  ambient  lighting  condition  may  not  be  satisfactory.  An  alter- 
nate approach,  shown  in  Figure  4-11,  combines  the  electronic  CRT  and  pro- 
jected film  images  at  a combining  glass.  An  important  advantage  of  this 
approach  is  that  the  two  images  can  be  separately  enhanced  before  combina- 
tion. The  CRT  image  is  monochromatic  (P-43  green  phosphor)  and  is  en- 
hanced by  using  a narrow  band  pass  filter  over  the  CRT  face.  The  film  image 
is  fully  chromatic  and  is  enhanced  by  using  a high  contrast  screen  matoi  al 
in  conjunction  with  a circular  polarizer.  Antireflective  coatings  are  used 
throughout.  The  result  is  a full- color  map  readable  under  high  ambient 
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EQUIVALENT  WIDTH  ON  CHART  = 1.70  X 12/1.3  = 15.7  IN.  (30  OVERMAG.) 
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1 : 1 M 78.8  NMI  5,240  X 5,240  NMI  215  X 12,600  NMI 

1:1/2 M 34.4  NMI  2,620  X 2,620  NMI  108  X 6,300  NMI 


Figure  4-9.  Film  Coverage 
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illumination.  The  combined  image  is  real,  not  virtual;  and  allows  consider- 
able freedom  of  head  movement  for  the  pi  lot  or  observer. 

The  CRT  and  map  image  planes  are  spaced  optically  equidistant  from  the 
beam  splitting  mirror  surface.  The  observer  sees  two  images  which  appear 
to  be  combined  at  one  image  plane,  the  CRT  face  plane.  This  particular  CRT 
presentation  has  a rectangular  format  15.2  cm  high  and  13  cm  inches  wide 
and  is  proposed  for  use  by  the  U.  S.  Navy  on  the  F-18  aircraft  simulator®. 

The  dimensions  of  the  display  unit  are  estimated  to  be  17.  3 cm  wide  by  26  cm 
high  and  51  cm  long. 

The  disadvantage  of  this  approach  is  that  direct  sunlight  can  impinge  on  the 
CRT  face  and  the  map  projection  screen,  reducing  the  display  contrast.  The 
alternate  approach  shown  in  Figure  4-12  overcomes  the  problem  by  presenting 
the  image  via  a viewing  system  consisting  of  a combiner  mirror,  transfer  lens 
and  field  lens.  The  combiner  mirror  reflects  the  real  map  image  (from  the 
rear  projection  screen)  and  transmits  the  CRT  image  such  that  they  appear 
superimposed  in  the  same  plane.  To  prevent  paralax,  it  is  important  that 
the  relative  positions  of  the  CRT  screen,  projection  screen  and  combiner 
all  be  held  to  close  tolerances.  The  exit  pupil  of  this  particular  system®  is 
19  cm  in  diameter  at  a viewing  distance  of  66  cm.  This  allows  a vertical 
head  movement  of  plus  or  minus  9.  5 cm  and  a horizontal  head  movement  of 
about  ±6.  3 cm,  depending  upon  the  interpupilary  distance  of  the  operator. 

The  field  lens  viewing  system  has  several  important  advantages  over  the  sys- 
tem shown  in  Figure  4-11.  Note  that  the  transfer  lens  focuses  a real,  com- 
bined image  at  the  position  of  the  field  lens.  The  observer  then  sees  the  real 
image  in  the  field  lens  and  the  lens  directs  the  diverging  light  from  the  trans- 
fer lens  to  the  exit  pupil  for  increased  brightness. 

The  advantages  of  this  system  are  several.  First,  direct  sunlight  cannot 
enter  the  system  to  reduce  the  contrast  of  either  the  map  or  CRT  images. 
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lo  impinge  upon  the  CRT  or  map  projection  screen,  light  must  pass  through 
the  transfer  lens.  However,  to  pass  through  the  transfer  lens  it  must  also 
pass  through  the  system  exit  pupil,  which  is  the  image  of  the  transfer  lens 
exit  pupil.  This  is  impossible  since  the  exit  pupil's  diameter  is  completely 
filled  by  the  operator's  head. 

Second,  the  display  has  higher  brightness.  The  brightness  of  the  primary 
image  (as  seen  by  the  operator)  is  the  same  as  the  brightness  of  the-  source, 
i.e.  . the  map  projection  screen  and  CRT  phosphor.  However,  these  images 
are  almost  five  times  brighter  (due  to  2.  2 times  image  magnification)  than  if 
full  sized  images  were  used.  This  assumes  that  the  brightness  of  the  map 
projection  screen  is  inversely  proportional  to  its  area,  i.e.  , 6.9  cm  map 
screen  diameter  instead  of  a 15.  2 cm  map  diameter.  The  brightness  of  the 
i 6 cm  CR  I used  in  the  system  has  twice  the  brightness  of  a 15.  2 cm 
diameter  CRT  for  equivalent  drive  powers.  However,  a 7.  6 cm  CRT  has 
tour  times  the  brightness  of  a 15.2  cm  rear  port  CRT  since  rear  port  CRTs 
cannot  have  aluminized  screens. 

This  particular  system  accommodates  17.  37  m of  35  mm  film.  This  is 
equivalent  to  an  area  of  366  km  by  366  km  with  a film-to-screen  magnifica- 
tion of  7.  5 times  and  a chart-to-film  reduction  factor  of  15.  This  system  has 
an  access  time  of  10  seconds  and  the  ground  area  is  split  into  E-W  strips 
photographed  sequentially  along  the  length  of  the  film.  Different  scales  are 
located  on  different  parts  of  the  film,  permitting  a variable  magnification. 

The  accuracy  of  synchronizing  the  map  to  the  CRT  images  is  approximately 
one  percent  anywhere  on  the  screen^6^-  E) 

All  moving  map  systems  accept  a manual  initialization  to  the  approximate  air- 
craft location  and  heading.  The  projected  map  image  can  then  be  compared 
to  the  radar  map  image  for  updating.  When  the  map  image  is  in  manual 
registration  with  the  radar  map.  the  map  drive  control  can  be  switched  over 
to  automatic  update  by  the  aircraft  digital  navigational  control.  These  systems 
are  capable  of  selecting  scales  to  match  the  radar  map. 
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Displays 

Aircraft  Display  Requirements  and  Display  Types  --  The  primary  require- 
ment for  any  cockpit  display  is  that  it  must  be  legible  under  high  and  low 
levels  of  illumination,  as  well  as  in  darkness,  and  to  pilots  of  all  age  groups. 

Illumination  levels  in  direct  sunlight  can  be  as  high  as  10  lm/m  . Increased 
reliability  and  ruggedness  beyond  that  typical  of  incandescent  lamps  and 
electromechanical  devices  is  sorely  needed. 

It  is  also  desirable  for  displays  to  have  a graphic  (line-drawing)  capability 
and  be  able  to  show  alphanumeric  characters  and  symbols  without  a large 
number  of  wires  connecting  the  display  to  the  drive  electronics.  A rcraft 
displays  must  also  be  compatible  with  modern  integrated  circuits  and  should 
operate  on  low  voltage  aircraft  power.  The  display  should  also  operate  over 
a minimum  temperature  range  of  -20  to  +70°C. 

Displays  may  be  classified  into  one  of  two  types:  passive  displays,  which 
function  by  modulating  ambient  illumination;  and  emissive  or  active  displays 
which  generate  light.  The  following  paragraphs  briefly  describe  promising 
examples  of  both  types  and  give  their  general  characteristics;  more  technical 
details  are  supplied  in  Appendices  I through  P of  reference  7. 
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The  introduction  of  on-board  computing  systems  means  that  more  sophisticated  4 

displays  can  be  made  available.  For  example,  an  on-board  computing  system 
can  directly  indicate  through  the  display  the  estimated  time  of  arrival  at  any 
particular  point  instead  of  requiring  the  pilot  to  read  several  dials  and  per- 
form a calculation.  Displays  can  also  present  navigational  information  data 
on  the  mechanical  state  of  the  aircraft,  and  results  of  any  automatic  checks 
carried  out  by  the  calculator.  It  is  quite  possible  that  one  or  more  larger 
displavs  may  be  used  to  perform  these  functions,  reducing  the  cockpit  area 
taken  up  by  instrumentation. 


Contemporary  electromechanical  instruments  are  bulky,  require  large  panel 
areas,  and  are  undesirably  heavy.  They  also  have  a relatively  slow  response, 
contain  moving  parts,  and  may  be  sensitive  to  vibration.  The  only  other  dis- 
play technique  commonly  used  is  the  cathode  ray  tube  (CRT);  however,  CRTs 
are  rather  heavy  and  bulky,  require  a high  voltage,  and  carry  a slight  risk  of 
implosion. 


Characteristics  of  Passive  Displays  --  The  advantage  of  reflective,  passive 
displays  is  that  their  contrast  is  independent  of  the  illumination  level,  i,  e.  . 
tliey  modulate  the  ambient  light.  This  modulation  is  controlled  on  individual 
elements  to  form  the  desired  image.  They  consume  little  power  and  are 
ideal  for  large  displays,  A difficulty  with  passive  displays  is  material  satu- 
ration with  increasing  drive  voltages.  It  is  imposs ible  for  the  passive  display 
to  compensate  for  the  necessary  short-address  period  by  driving  the  display 
harder  at  higher  voltages.  To  fill  this  need,  some  form  of  either  inherent 
(in  the  form  of  high  decay-time /rise -time  ratio)  or  on-site  memory  must  be 
employed. 


Passive  displays  also  require  some  illumination  for  night  viewing  and  their 
field  of  view  may  be  limited.  An  additional  possible  disadvantage  in  air- 
borne application  is  a degradation  of  contrast  in  diffuse  illumination,  i.  e.  , 
fixing  through  a cloud,  for  those  displays  based  on  a scattering  mechanism. 


Characteristics  of  Emissive  Displays  --  Light- emitting  displays  are  less 

critically  dependent  upon  the  color,  reflectivity,  and  position  of  surrounding 

objects  than  reflective  displays  and  are  more  easilx  read  in  low  ambient 

lighting  conditions.  However,  they  must  have  sufficient  luminance  to  com- 
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pete  with  ambient  illumination  (up  to  10  lm/m  ) must  be  visible  when  the 
eye  is  adapted  to  luminance  levels  up  to  3.  4 x 10^  lm/sr/m^  and  must  have 
adequate  contrast  with  the  immediate  surrounding  cockpit  area. 
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Several  techniques  are  available  to  reduce  the  amount  of  ambient  light  reaching 
the  display.  Bezels  can  be  used  where  a limited  field  of  view  is  acceptable 
and  the  space  available.  More  compact  methods  restrict  the  range  of  incident 
light  on  the  display  through  the  use  of  louvered  plastic  or  imbedded  mesh 
filters.  Alternately  neutral  density,  colored,  or  circularly  polarizing  filters 
can  be  used.  The  colored  filter  matches  peak  spectral  emittance  of  the  displav. 
All  filters  attenuate  the  incident  ambient  illumination  twice  while  they  attenuate 
the  emitted  display  light  once,  resulting  in  improved  contrast. 

One  important  advantage  of  the  emissive  display  is  an  unsaturated  output. 

The  emitted  light  increases  in  proportion  to  the  drive  voltage  up  to  a relatively 
high  level.  However,  the  display  luminance  must  be  reduced  as  emitted  dis- 
play light  that  is  bright  enough  under  high  illumination  conditions  will  be  too 
bright  and,  therefore  unreadable  at  low  luminance  levels.  The  disadvantage 
of  the  emissive  display  is  the  high  power  consumption  that  results  from  the 
low  emission  efficiencies. 

Cathode  ray  tubes  (CRT)  are  currently  the  dominant  electronic  displays,  ex- 
cept for  pure  alphanumerics.  They  arc  the  standard  of  performance  to  which 
all  other  displays  are  compared.  The  CRT  is  a vacuum  tube  with  a cathode 
luminescent  phosphor  that  is  bombarded  by  a position-controlled  electron 
beam.  Operation  is  generally  either  video  raster-scan  or  random  access. 

In  the  raster-scan  mode,  the  beam  traverses  the  whole  area  of  the  display, 
line-by-line.  Display  brig htness  is  determined  by  the  beam  power,  phosphor 
efficiency,  and  duty  cycle.  The  random-access  mode  is  used  with  computer- 
generated nonvideo  signals,  typically  alphanumeric  or  graphic  information, 
which  is  usually  generated  with  a stroke  writing  technique. 

The  CRT  is  not  good  for  general  aircraft  because  of  the  large  ratio  of  tube- 
depth  to  viewing-facc-diameter  the  high  voltages  required  for  adequate 
brightness,  the  associated  electronic  circuitry,  and  the  bulk  and  power 
requirements. 
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The  basic  advantage  of  the  CRT  is  the  high  peak  brightness  of  greater  than 
2 

three  million  cd/m  . which  permits  a high  average  brightness  at  TV  display 
rates.  The  primary  concern  for  television  is  the  low  duty  cycle  of  typically 
1/500  or  1/1000.  In  spite  of  this  low  duty  cycle,  there  are  displays  with 
luminance  levels  of  10,200  cd/m  with  resolution  in  excess  of  40  lines/mm. 
The  CRT  can  achieve  these  luminance  levels  due  to  a very  high  instantaneous 
luminance  and  a decay  time  that  is  long  relative  to  the  frame  rate. 

Another  display  requirement  that  is  fulfilled  by  the  CRT  is  the  uniformity  of 
both  the  threshold  voltage  and  output  beyond  the  threshold.  Without  uniformity 
there  is  an  undesirable  mottling  of  the  picture.  The  CRT  meets  the  uniformity 
requirement  as  there  are  more  than  1000  phosphor  particles  within  the  elec- 
tronic scanning  beam;  this  averaging  gives  good  uniformity  of  luminance 
across  the  face  of  the  display.  The  redundancy  of  the  display,  i.  e.  , multiple 
elements  within  the  scanning  beam,  also  gives  a high  degree  of  reliability. 

For  a material  to  be  considered  for  a display  it  must  not  only  have  the  capa- 
bility of  high  peak  brightness  but  also  must  have  intrinsic  memory  or  per- 
sistence. The  CRT  meets  both  these  requirements. 

One  of  the  CRT's  major  advantages  is  cost,  which  is  2 3 cents  per  character 
(Table  4-2).  Further  development  is  being  funded  and  is  inherently  low  in 
technical  risk. 


Table  4-2.  Display  Subsystem  Projected  Prices* 


Number  of 
Characters 

Gas 

Plasma, 

y 

Gas 

Discharge, 

CRT, 

$ 

Liquid 

Crystal, 

LED's 

$ 

32 

6.  05 

5.  29 

3.  56 

256 

3.  75 

1.  09 

1.  10 

512 

2.  33 

0.  78 

0.  66 

0.  40 

0.  75 

1024 

1.  69 

0.  62 

0.  35 

2 048 

0.  97 

0.  56 

0.  23 

Original  equipment  manufacturer  price  per  character  including  decode 
drive  and  refresh  buffer. 
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A typical  aircraft  CRT  display  for  an  imaging  forward-looking  infrar-  or 

2 

has  a mean  luminance  of  171  cd/m  , with  a peak  luminance  of  680  ed/i; 
and  requires  75  watts  of  power.  The  display  is  11  x 8 cm  and  has  an  800  line- 
per-picture  height  resolution.  Envelope  dimensions  are  19x17x2  8 cm.  Typ- 
ical mean  luminance  levels  for  alphanumeric  CRT  displavs  will  run  as  high 
2 

as  1710  cd/m  . The  cost  of  a commercial  TV  display  is  $100  and  a MIL  spec 
unit  will  cost  between  $4000  and  $5000. 

The  size  of  a CRT  suggests  using  flat  panel  displavs.  Unfortunately,  existing 
flat  panel  materials  are  not  bright  enough  without  increased  persistance 
(inherent  memory)  or  creating  persistence  by  use  of  electronic  on-site  mem- 
ory. Consequently,  the  problems  and  requirements  of  matrix-addressed  dis- 
plays with  intrinsic  or  on-site  memory  must  be  considered. 

Matrix-Addressed  Displays  --  Today,  the  potential  for  flat- panel  video  tech- 
nology is  limited  to  liquid  crystal,  gas-discharge,  and  possibly  electrophoretic 
devices.  Little  effort  has  been  devoted  to  the  problem  of  addressing  or  signal 
distribution  in  multi-element  displays  as  compared  to  the  development  of  novel 
display  materials.  Therefore,  the  major  current  problem  is  getting  the  dis- 
play message  to  all  points  on  the  screen;  the  display  material  itself  becoming 
a secondary  problem. 

It  is  necessary  to  distinguish  between  active  and  passive  matrix  addressing. 
Active  matrix  addressing  is  defined  as  containing  gain-producing,  switching, 
and/or  memory  elemerts  at  every  display  cell.  A passive  matrix,  in  con- 
trast, is  normally  composed  of  two  sets  of  parallel  conductors  oriented  at 
right  angles  to  each  other,  with  the  display  medium  sandwiched  in  between. 

The  major  problems  associated  with  passive  matrix  driven  displays  are: 

1)  the  excessive  demands  placed  on  the  electrical  and  optical  characteristics 
of  the  display  material,  and  2)  the  complexity  and  cost  of  the  external  drive 
electronics.  All  of  these  problems  can  be  solved  or  bypassed  by  the  develop- 
ment of  active  matrices. 
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The  function  of  the  active  matrix  is  to  integrate  the  electronic  functions  and 
to  compensate  for  the  deficiencies  in  the  electro-optical  characteristics  of 
the  chosen  display  material.  Liquid  crystal  displays,  for  example,  have 
slow  response  times  and  electroluminescent  displays  require  comparatively 
high  voltages  and  lack  inherent  memory.  Field-effect  liquid  crystal  panels, 
with  their  own  capacitance  and  high  impedance,  form  their  own  memory. 
Consequently  only  a transistor  is  required  at  each  display  element  to  gate 
the  initial  charge  (Figure  4-13),  Electroluminescent  displays,  on  the  other 
hand,  of  either  a-c  or  d-c,  require  a more  complex  circuitry  due  to  high 
voltages  and  significant  current  levels.  (Compare  the  circuits  in  Figures  4-13 
and  4-14. ) 

Most  flat  panel  display  schemes  use  a matrix  arrangement  in  which  the  in- 
dividual display  elements  are  connected  between  row  and  column  electrodes. 

All  elements  along  a row  and  column  are  partially  stimulated  when  a single 
element  is  addressed  at  the  intersection  of  a row  and  a column  because  all 
elements  of  the  matrix  are  permanently  interconnected.  Therefore,  most 
elements  will  be  partially  excited  (cross  talk)  many  more  times  than  directly 
excited.  Each  element  must  have  a sharp  threshold  above  the  partially  ex- 
cited level  to  have  adequate  contrast.  An  additional  diode  is  required  in  dis- 
play media  not  having  this  inherent  sharp  threshold. 

Matrix-address  arrays  are  generally  operated  a line  at  a time  such  that 
each  element  in  the  line  can  be  driven  for  a full  line  time  rather  than  for  just 
one  element  time.  This  increases  excitation  time  by  a factor  of  the  number 
of  elements  in  a horizontal  line,  increasing  the  output  brightness  of  the  display 
as  much  as  500  times. 

The  longer  excitation  time  relieves  the  drive  electronics  of  handling  short, 
high-powered  pulses  to  a high-capacitance  electrode.  However,  line-at-a- 
time  operation  at  TV  rates  requires  a 15-kHz  shift  register  for  the  horizontal 
rows,  a video  sampler-distributor,  two  storage  registers  and  a column 
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Figure  4-14.  Design  of  Elemental  Matrix  Circuit 
for  the  6 by  6 Inch.  20  I.ines/Inch 
EL  Display 


driver  (see  Figures  4-15  and  4-16).  The  video  signal  for  one  line  of  infor- 
mation is  fed  into  the  sampler-distributor.  Parallel  video  information  is  then 
stored  in  one  register  while  the  other  storage  register  is  feeding  the  column 
drivers.  The  storage  registers  and  column  drivers  may  be  either  analog  or 
digital,  but  it  is  difficult  to  produce  a large  number  of  analog  circuits  with 
the  same  characteristics  thereby  avoiding  vertical  streaks  in  the  picture. 
Thus,  digital  circuits  are  preferred  to  overcome  this  difficulty. 

The  average  brightness  of  matrix  displays  developed  to  date  is  not  as  great 
as  desired.  Table  4-3  lists  the  expected  maximum  brightness  for  various 
display  media  when  operated  in  a line-at-a-time  format.  This  problem  can 
be  eliminated  by  incorporating  an  analog  memory  driver  at  each  picture 
element  location  (Figure  4-16),  which  would  maintain  the  display  element 
excitation  during  the  whole  frame  time  and  increase  display  brightness 
accordingly.  Moreover,  the  memory  elements  must  either  control  an 
external  power  source  or  act  as  amplifiers,  which  creates  problems  of 
element- to-element  uniformity. 

The  need  to  avoid  an  unmanageably  large  number  of  connections  and  to  provide 
integrated  electronic  circuitry  leads  to  the  large-scale- integration  (LSI) 
concept  of  the  entire  display.  Monolithic  integrated  circuits  of  matrices  over 
two  to  three  inches  may  be  beyond  the  present  or  near-future  capability  of 
silicon  technologies  (Ref.  12).  However,  thin-film  technology  appears  ade- 
quate for  large-area  active  matrices  and  can  be  integrated  with  a large 
variety  of  display  media.  Thin-film  transistor  work  is  currently  being 
conducted  by  Westinghouse  and  Aerojet  General  to  build  large  active 
matrices. 

It  is  not  clear  which  technology  thin-film  transistors  (TFTs)  or  silicon 
processes,  will  eventually  produce  usable  matrix  displays.  A one-inch 
square  silicon  matrix  is  huge  by  silicon  integrated  circuits  (IC)  standards; 
shorting  problems,  insulator  pin  holes . metalization  wafer  breakage, 
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Figure  4-15.  Block  Diagram  of  Matrix  TV  Display 

(Diode  shown  can  be  inherent  feature 

of  display  elements  (V  > V ) 
v J so 
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Figure  4-16.  Circuit  Configuration  \eedcd  for  Frame 
Storage  in  Matrix  Display 
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Table  4-3.  Average  Brightness  and  Luminous 
Efficiency  of  Display  Media 


Display  Type 

Peak 

Brightness 

(Cd/m2) 

Maximum  Average 
Brightness  at  1/5000 
Duty  Factor 
(Cd/m2) 

Efficiency 

(Im/w) 

Plasma 

7500 

10-15 

0.  5-1 

AC  Electroluminescent 

500 

1-5 

5-10 

DC  Electroluminescent 

3000-5000 

25-50 

0.  5-1 

Light  Emitting  Diodes 

5000 

5-10 

0.  5 

Catholuminescent 

phosphors 

> 100.000 

>500 

100 

in-process  inspection  and  testing  are  the  problems  encountered  with  obtaining 
the  needed  crystalline  perfection  over  large  areas. 


The  key  problem  in  thin-film  technology  is  the  need  for  a 100  percent  area 
yield  in  all  process  steps.  This  is  different  than  the  semiconductor  circuit 
manufacture  where  many  chips  are  formed  simultaneously  on  a single  wafer 
and  100-percent  wafer  yields  are  neither  needed  nor  expected.  The  thin-film 
deposition  sequence  is  long,  somewhat  complex,  and  subject  to  operator 
errors.  Not  only  mask  fabrication  (for  thin-film  deposition)  but  also  the 
alignment  during  the  deposition  steps  must  be  perfect.  However,  mask  loca- 
tion changing,  deposition  material  sequencing,  and  thickness  monitoring  can 
all  be  done  by  automatic  equipment.  Also,  the  process  steps  are  quite  fast, 
therefore,  a large  output- per- machine  can  be  expected  with  reasonable  yields 
as  manual  operations  are  eliminated. 


Passive  Display  Material  Types  --  Liquid  crystal,  electrochromic . electro- 
phoretic, and  PLZT  are  four  examples  of  low-power  passive  displays.  These 
displays  do  not  emit  visible  radiation;  rather  they  control  the  passage  of 
externally  generated  light  through  the  display.  Structurally,  these  four 
displays  have  similar  principal  operations  but  dissimilar  properties. 

Liquid  Crystal  Displays  (LCDs)  --  Liquid  crystal  materials  have  the 
normal  solid  and  isotropic  liquid  phases  of  normal  liquids,  but  they  also  have 
a third  phase  which  occurs  between  the  solid  and  the  isotropic  liquid  phase. 

In  this  intermediate  phase,  the  liquid  crystal  flows  like  a fluid  but  exhibits  a 
crystalline  organic  state.  The  molecules  are  usually  long  and  rod- like,  and 
are  responsible  for  the  display- related  anisotropic  properties. 

The  liquid  crystal  is  confined  between  two  glass  plates  with  their  conductive 
coatings  in  contact  with  the  material.  The  electro- optic  phenomena  can  be 
divided  into  two  groups:  1)  those  caused  by  dielectric  forces  and  the  so- 
called  field  effects;  and  2)  those  induced  by  a combination  of  dielectric  and 
conductive  forces. 

The  most  promising  material  and  arrangement  results  in  a cell  structure 
called  the  twisted  nematic.  Linear  polarized  light  propagating  perpendicular 
to  the  cell  is  rotated  approximately  90  degrees.  Maximum  light  transmission 
is  obtained  by  orienting  a crossed  polarizer  and  analyzer.  The  transmitted 
light  decreases  when  the  applied  voltage  exceeds  a certain  threshold  voltage 
and  the  liquid  crystal  molecules  start  to  change  their  orientation  and  the 
polarization  of  the  display. 


The  advantages  of  LCDs  are  a low  voltage  and  low  current,  which  allow  the 
cell  to  be  driven  by  CMOS  ICs  and  the  fact  that  their  contrast  is  independent 
of  ambient  light  levels.  The  disadvantages  are  a slow  response  time  (10  to 
100  msec),  and  the  difficulty  of  multiplexing.  They  have  a limited  operating 
temperature  range,  and  the  viewing  angle  affects  the  contrast. 


Electrochromic  Displays  (ECDs)  --  An  electrochromic  material  is  one 
in  which  the  color  is  changed  by  an  electric-field  current.  One  version  uses 
an  aqueous  solution  of  a special  organic  dye  contained  between  a pair  of  trans- 
parent electrodes.  This  display  relies  upon  the  oxidation- reduction  reaction 
of  the  dye  controlled  by  the  application  of  voltage  to  the  electrodes.  The  dye 
is  colorless  in  the  oxidized  state.  Application  of  about  two  volts  across  the 
electrodes  reduces  reactions,  resulting  a dramatic  change  in  color.  When 
the  polarity  of  the  voltage  is  reversed,  the  color  compound  oxidizes  back  to 
the  colorless  state.  Another  type  of  ECD  uses  a solid,  inorganic  film  as  the 
electrochromic  material.  A glass  substrate  with  a transparent  conducting 
layer  or  electrode  is  coated  with  a thin  film  of  the  material.  A layer  of 
insulating  material  is  deposited  over  the  film,  followed  by  a second  electrode. 
The  film  is  colorless  in  the  normal  state,  but  when  a voltage  is  applied  the 
display  appears  blue.  When  the  polarity  is  reversed,  the  film  again  becomes 
colorless. 

The  advantages  of  electrochromic  material  are  that  it  operates  on  five  volts 
or  less,  and  its  appearance  does  not  change  with  the  viewing  angle.  Its  dis- 
advantages are  a slow  response  time  (between  20  to  200  msec)  and  difficulty  in 
multiplexing. 

Electrophoretic  Displays  --  This  type  of  display  uses  pigment  particles 
of  one  color  suspended  in  a liquid  of  a different  color.  The  suspension  is 
sandwiched  between  a pair  of  electrodes,  at  least  one  of  which  is  transparent. 
The  pigment  particles  are  held  in  a colloidal  suspension  and  carry  a charge. 
When  the  electrodes  are  charged  the  pigment  particles  move  toward  the  front 
electrode,  where  they  collect  to  scatter  the  ambient  light,  making  the  display 
change  color.  Reversing  the  polarity  causes  the  particles  to  move  towards 
the  ouver  electrode,  changing  the  color  of  the  display  to  that  of  the  suspension 
fluid. 
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These  displays  operate  at  low  voltages  (less  than  10  volts)  and  are  legible 
over  a wide  range  of  viewing  angles.  On  the  other  hand,  thev  (iave  a very  long 
response  time  (in  100's  of  msec)  and  the  particles  may  eventually  settle  out  of 
suspension  or  may  be  damaged  by  shock  and  vibration. 

Pl.ZT  Displays  --  PLZT  is  a transparent  ceramic,  electro-optical 
material  with  voltage-dependent  transmission  properties,  because  the  mate- 
rial is  in  the  solid  state,  it  is  necessary  to  use  the  optical  properties  of 
induced  birefringence  and  of  scattering.  The  birefrmgent  mode  is  imple- 
mented by  operating  the  material  such  that  the  applied  voltage  will  vary  the 
polarization  of  the  transmitted  light.  The  effect  can  be  observed  through  an 
analyzer  sheet  as  a change  in  intensity.  In  the  scattering  mode,  the  trans- 
mitted light  is  diverted  from  the  normal  propagation  direction  and  scattered 
into  a larger  solid  angle  by  the  application  of  the  voltage  to  the  material. 

Within  these  two  modes  of  operation,  the  material  can  further  be  classified 
according  to  its  memory.  The  material  may  either  have  memory  or  none  in 
either  mode  of  operation. 

A significant  advantage  of  the  material  is  its  potential  for  intrinsic  memory, 
which  would  permit  displays  with  high  contrast.  However,  its  disadvantages 
of  a high  drive  voltage  (40  volts),  cracking  under  stress,  poor  contrast,  and 
small  viewing  angles  are  also  significant.  The  requirement  for  transverse 
excitation  •equires  either  depositing  interdigitized  electrodes,  or  cross- 
slotting  the  material  itself.  Both  techniques  are  difficult  for  matrix-cell 
fabrication. 

Kmissivc  Displays  Material  Types  --  Three  examples  of  emissive  displavs 
are  light  emitting  diode  (LED),  gas  plasma  devices  (GPD) . and  electro- 
luminescent devices  (ELD).  These  devices  emit  visible  radiation  and. 
consequently  require  more  power  than  the  passive  display. 
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Light  Emitting  Diode  — This  device  is  essentially  an  electro- optical 
transistor,  Basic  components  are  the  diode  leads,  crystal  chip,  and  possibly 
a diffuser  lens.  Diode-chip-emitted  light  is  proportional  to  the  current  flow. 
Luminous  levels  up  to  340-1400  Cd/m  are  common  without  heat  sinking  and 
cooling.  Voltage  levels  vary  from  1.  4 to  4.  5 volts,  depending  upon  the 
material.  Color  of  the  device's  output  is  primarily  red.  but  green  and  yellow 
are  available  and  a blue  output  is  under  development.  LEDs  are  high-current, 
low-voltage  devices  that  are  compatible  with  discrete  MOS  transistors. 

The  advantages  of  LEDs  are  that  they  can  be  molded  into  a large  variety  of 
shape6  with  a large  range  of  optics.  They  are  compatible  with  the  low-voltage 
supply  provided  by  conventional  transistor  circuits.  They  are  rugged,  have  a 
high  reliability  with  a long  life,  and  are  resistant  to  temperature  change.  The 
disadvantages  are  that  the  material  is  relatively  expensive  and  the  LED  is  not 
readily  driven  to  maximum  brilliance  by  MOS  ICs,  particularly  CMOS  ICs, 
due  to  the  electrical  current  requirements.  Consequent^,  they  are  not  likely 
to  be  used  in  large- matrix-driven  displays. 

Gas- Plasma  Devices  (GPP)  — A plasma  panel  is  a gas-filled  device.  It 
is  the  most  advanced  class  of  all  flat-panel  displays.  It  has  two  modes  of 
operation:  ac  and  dc.  The  gas  fill  is  usually  neon  or  a mixture  of  noble  gases 
with  other  inert  gases,  such  as  nitrogen.  A panel- resolution  element  is  defined 
by  a very  small  gas  volume  between  orthogonally  oriented  sets  of  parallel 
conductors,  one  on  the  front  transparent  surface,  and  the  other  on  the  back 
transparent  surface.  The  visible  plasma  discharge  appears  when  a high- 
voltage,  low-current  charge  is  applied  between  the  electrodes.  Some  of  the 
many  display  types  are  self  scanning  and  require  few  electrical  connections. 
These  displays  have  been  under  very  intensive  study  to  develop  real-time  T\ 
displays  as  they  are  closest  tc  having  all  the  required  characteristics. 
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The  advantages  of  d-c  GPDs  are  that  they  have  sharp  thresholds  with  a high 
contrast  ratio.  They  also  permit  the  use  of  a gray  scale.  The  response 
time  is  fast  enough  for  line-at-a-time  addressing.  The  disadvantages  are  that 
they  primarily  come  in  lhe  color  red,  require  170  volts,  and  have  no  inherent 
memory. 

Electroluminescent  Displays  (ELD)  --  These  devices  are  essentially  an 
electrical,  luminous  capacitor  consisting  of  two  flat  electrodes,  one  of  which 
is  transparent,  enclosing  a space  filled  with  an  electroluminescent  phosphor. 
When  a low-current,  high  voltage  is  applied  to  the  electrodes,  a field  is 
created  causing  the  coated  areas  to  emit  light.  Most  displays  are  single 
colored,  usually  green  or  orqnge  because  of  the  higher  efficiencies.  The 
potential  may  be  ac  or  dc,  depending  upon  the  structure,  but  they  usually 
are  ac. 

The  advantages  of  these  devices  are  that  they  can  be  driven  by  TFTs  for  large- 
scale  matrix  displays.  They  are  lightweight,  have  low  current  requirements, 
and  produce  no  heat.  They  have  a high  reliability  and  a long  life  and  are  not 
affected  by  temperature,  vibration,  or  shock.  Luminance  levels  are  usually 
only  15  to  100  Cd/m  and  they  come  in  only  one  color  for  a particular  dis- 
play type,  but  that  may  be  red,  green,  yellow,  blue,  or  white.  The  disadvan- 
tage is  the  high  voltage  requirement  (up  to  600  volts). 

Display  Material  Comparison  --  The  basic  parameters  for  comparing  candi- 
date display  materials  are:  visibility  in  the  high  ambient  illumination  of  the 
cockpit,  luminance,  contrast,  life,  power  requirements,  and  resolution. 

Size  and  bulk  are  not  compared  as  the  materials  are  all  intended  for  flat  panel 
displays,  and  it  is  assumed  that  panel  depth  will  not  be  a problem  as  drive 
electronics  and  probably  be  miniaturized.  If  the  bulk  of  the  electronics 
becomes  a problem  for  high-voltage  displays,  they  can  be  mounted  in  a 


Table  4-4  presents  display  material  characteristics,  and  a detailed  discus- 
sion of  these  parameters  is  presented  in  Appendices  I through  P of  Ref.  7. 

The  pacing  requirements  for  a display  material  are  contrast  and  luminance. 

A specific  case  for  these  requirements  is  presented  in  Table  4-5.  A discus- 
sion of  these  requirements  and  the  deviation  from  them  is  presented  in  the 
attached  Appendix  A.  Because  data  on  reflectance  is  not  available  in  the 
literature,  five  percent  reflectance  is  assumed  for  all  materials.  A display 
or  time-averaged  contrast  of  10:1  is  also  assumed  as  a requirement.  The 
intrinsic  material  contrast  and  luminance  is  equal  to  the  time-averaged 
contrast  and  luminance  only  when  the  display  elements  are  on  continuously 
(nonmultiplex  or  memory  matrix  addressed).  However,  the  contrast 
luminance  is  reduced  when  a character  is  multiplexed  or  a matrix  is  addressed 
one  line  at  a time  without  memory. 

It  should  be  noted  that  passive  display  luminance  is  a function  of  reflected 
ambient  illumination.  It  is  assumed  that  the  cover  glass  reflectance  is  lower 
than  the  display  reflectance  and,  therefore,  display  contrast  remains  con- 
stant with  illumination. 

Table  4-5  presents  the  contrast  and  luminance  requirements  for  three  condi- 
tions of  high  ambient  illumination  in  the  aircraft  cockpit.  The  first  condition 
is  when  the  display  is  shielded  from  all  ambient  illumination,  and  the  required 
display  luminance  is  established  by  eye  adaption  to  bright  white  clouds.  The 
second  condition  is  when  the  display  is  shielded  from  direct  sunlight  but  is 
illuminated  by  reflected  light  from  white  clouds  through  the  cockpit  windows. 
The  last  condition  of  direct  sun  illumination  is  the  most  difficult  for  active 
display  visibility.  This  table  does  not  include  the  use  of  ambient  light  sup- 
pression techniques  for  active  displays  because  specific  display  reflectance 
data  is  not  available.  Therefore,  Table  4-5  presents  a worst-case  set  of 
requirements  and  provides  a basis  for  a relative  comparison  of  display 
materials. 
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Tabic  4-4.  Display  Material  Characteristics 
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Table  4-4.  Display  Material  Characteristics  (Concluded) 
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Table  4-5  shows  that  all  passive  display  materials  have  adequate  intrinsic 
contrast  (10:1)  for  nonmultiplex  or  memory  matrix  address  displays.  All 
display  materials  except  electrochromic  (ECD)  and  perhaps  electrophoretic 
(EPID)  have  enough  intrinsic  contrast  (53:1)  for  use  as  16-character  line- 
multiplexed  or  nonmemory  (lme-at-a-time  address)  matrix  display.  However, 
there  are  no  materials  that  have  adequate  intrinsic  contrast  when  a high 
resolution  (100  to  500  characters/ line)  nonmemory  display  is  required. 

PLZT  may  have  a future  potential  of  1000:1  contrast,  but  it  requires  a high 
drive  voltage.  On-site  memory  is  a better  approach  to  the  development  of 
high-resolution  passive  displays  as  any  material  with  lower  intrinsic  contrast 
but  superior  electrical  characteristics  can  be  used. 

Intrinsic  contrast  requirements  for  active  displays  become  much  higher  than 
for  passive  displays  when  the  active  display  is  in  ambient  illumination 
(Appendix  A).  These  ratios  range  from  530;1  to  17,000:1,  and  none  of  the 
materials  meet  these  requirements.  On-site  nonlinear  devices  could  be  used 
to  improve  the  material  excitation  threshold  (Appendix  A)  and  contrast;  how- 
ever on-site  memory  can  be  provided  as  easily  and  then  high  contrast  would 
no  longer  be  required.  It  should  be  noted  that  intrinsic  luminance  require- 
ments for  active  displays  in  ambient  illumination  (Table  4-5)  becomes  much 
larger  than  the  capabilities  of  the  display  materials.  Even  two  orders  of 
magnitude  reduction  in  luminance  requirements  (Table  4-5)  for  high  resolution 
displays  through  the  use  of  ambient  light  suppression  techniques  will  not  help. 
Consequently,  an  on-site  memory  is  definitely  required  for  both  passive  and 
active  high-resolution  matrix  displays. 

Passive  Display  Status  --  The  most  promising  passive  display  material  is 
liquid  crystal  with  an  on-site  memory  (see  Appendix  I of  Ref.  7).  which  is 
being  developed  by  both  Hughes  Aircraft  and  Westinghouse.  Currently,  there 
appears  to  be  enough  development  funding  for  military  applications. 
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The  next,  most  promising  class  of  passive  dispLay  materials  are  the  electro- 
phoretics  (EPIDs),  with  the  colloidal-sized  T1C>2  particles  (Appendix  K of 
Ref.  7)  as  the  best  candidate.  As  there  does  not  appear  to  be  a specific  air- 
craft-oriented program  on  this  material,  there  is  more  risk  than  in  the  case 
of  I. CD  development. 

Active  Display  Status  --  CRTs  are  currently  the  best  device  for  aircraft 
displays  and  will  be  used  in  the  various  tradeoffs,  analyses  and  examples  in 
their  report.  However,  there  is  a high  probability  tliat  GPDs  with  a perfor- 
mance suitable  for  aircraft  displays  will  be  available  in  the  1980's.  Because 
of  the  current  effort  being  expended  on  GPDs  for  TV  use,  the  risks  involved 
are  the  suitability  of  the  physical  configuration  (size  mostly)  and  the  question 
of  how  much  ambient  light  suppression  and  shielding  will  be  required. 

I F.Ds  will  be  available  in  large  volume  and  many  types,  but  because  of  the 
relatively  high  drive  currents  required  for  high  brilliance,  there  are  some 
doubts  as  to  the  development  of  the  on-site  memory  necessary  for  high- 
resolution  matrix  displays.  There  is  no  doubt  as  to  the  availability  of  suit- 
able alphanumeric  displays. 

Long- Persistence  Phosphor  CRTs  --  Long-persistence  P-7  and  P-14  phosphors 
have  been  used  for  many  years  to  display  the  radar  and  1R  line-scan  imagery. 
The  disadvantages  of  using  long  persistence  tubes  are  a low  brightness  level, 
fixed  and  nonlinear  persistence,  and  an  inability  to  selectively  erase. 

'Hie  low  brightness  of  this  type  of  tube  makes  it  impossible  to  use  such  a 
display  in  aircraft  cockpits  without  a visor  or  hood  to  shield  ambient  light. 

Even  then,  the  operator  is  forced  to  adjust  quickly  from  exterior  ambient  light 
levels  as  high  as  10,000  foot  lamberts  to  the  low  brightness  of  CRT  long 
persistence  phosphor.  This  display  can  be  used  where  ambient  light  can  be 
controlled  and  the  long  persistence  CRT  provides  an  acceptable  display  for 
low  scan  rate  sensors. 
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Even  under  these  conditions,  the  image  smears,  if  the  image  from  one  frame 
is  not  completely  erased  by  the  time  the  new  image  is  painted  on  the  dispLay. 
However,  adjustments  to  the  display  to  promote  a rapid  fading  will  also  cause 
an  annoying  fade  rate  which  degrades  the  operator's  recognition  performance. 
Therefore,  other  displays  were  developed,  such  as  the  direct-view  storage 
tubes  (DVST). 

Direct-View  Storage  Tubes  (DVST)  --  A di.rect-view  storage  tube  display  pro- 
vides a bright,  nonflickering  image  with  a controlled  persistence.  The  high 
brightness  is  a result  of  a constant  source  of  electrons  to  illuminate  the  view- 
ing screen  phosphor  during  the  viewing  operation.  The  flooding  electron  beam 
is  modulated  by  the  stored  charge  pattern  on  dielectric  mesh  located  imme- 
diately in  back  of  the  phosphor.  The  picture  observed  on  the  face  (phosphor) 
of  the  tube  is  a projection  of  the  charge  pattern  on  the  dielectric  mesh.  This 
charge  pattern  is  written  on  the  dielectric  by  the  action  of  a focused  electron 
beam.  Both  conventional  and  fast- erase  storage  tubes  are  available.  A con- 
ventional DVST  requires  several  tenths  of  a second  for  erasure,  and  is  too 
slow  for  high-frame-rate  FLIR  or  LLLTV  sensors.  The  fast-erase  storage 
tube  can  be  erased  in  one  to  two  milliseconds,  and  therefore  presents  a bright 
TV  display. 

However,  fast-erase  on  a DVST  loses  resolution,  gray  scale  and  scan-to- 
scan  interference.  Also,  symbology  presented  on  either  a fast-erase  or  a 
conventional  DVST  smears  when  it  is  moved  across  a stored  image. 

Tvpical  characteristics  of  a fast-erase  DVST  are  100  stored  lines  per  inch  of 
resolution  (shrinking  raster),  10  percent  MPF  at  68  cycles /inch,  800-foot 
iamberts  of  brightness.  100-K  inch/second  writing  speed,  a storage  time  of 
10  seconds,  and  6 shades  of  gray. 
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Fast  erase  DVST  appears  most  applicable  where  the  radar  ground  mapping  is 
of  a poor  quality;  nonstored  symbology  is  not  required  simultaneously  with  a 
stored  image;  image  fade,  smear  and  gray  scale  is  not  important  (target 
recognition  is  not  required)  and  TV  image  brightness  is  more  important  than 
resolution  or  gray  scale.  Fast  erase  DVSTs  are  not  suitable  for  presenting 
E-O  sensor  imagery  for  a high  quality  multifunction  display.  However,  a 
conventional  DVST  can  display  radar  information  for  combining  with  E-O 
imagery  from  another,  more  suitable  display. 

Short  Persistence  Standard  CRTs  --  The  conventional  short  persistence  CRT 
is  the  best  choice  if  a single  indicator  is  to  be  used  for  a multifunction  display 
monitor.  However,  this  implies  the  requirement  for  some  sort  of  scan 
conversion  of  the  slow  scan  radar  sensor's  video.  The  short  persistence 
CRT  lias  a high  resolution  and  high  brightness.  It  also  has  good  visibility 
under  high  ambient  light  conditions  when  a narrowband  phosphor  and  a matched 
notch  filter  is  used. 

It  is  assumed  that  the  standard  TV  raster  video  scan  rates  is  the  best 
approach  for  the  aircraft  display.  However  annotation  of  display  with  alpha- 
numerics  requires  a selection  of  the  type  of  deflection  system,  ^here  appear 
to  be  three  options:  a wideband  deflection  system;  a dual  mode  deflec- 
tion system;  or  an  all- raster  energy  recovery  deflection  system. 

The  first  option  is  the  wideband  deflection  system,  which  is  capable  of  medium- 
speed  caligraphic  generation  of  synthetic  symbology  during  pure  symbol  modes 
and  also  raster  scanning  when  an  EO  sensor  imagery  is  selected.  When  a 
raster  sensor  is  displayed,  overlay  symbols  are  generated  during  the  raster 
vertical  retrace.  The  advantage  of  writing  a symbol  during  the  vertical 
retrace  period  is  that  the  brightness  of  the  symbol  adds  to  the  brightness  of 
the  image.  Therefore,  the  symbol  can  be  seen  even  when  superimposed  on  a 
peak-white  area  of  the  image.  In  other  words,  it  becomes  whiter  than  white. 
This  ensures  that  the  synthetic  data  is  seen  everywhere  with  good  contrast 
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for  the  image  picture.  The  disadvantages  of  the  wideband  deflection  amplifier 
are  the  excessive  power  requirements  and  heat  dissipation.  Another  dis- 
advantage is  the  complexity  of  the  wideband  deflection  system  created  by  the 
relative'y  high  writing  speeds  associated  with  the  sequential  construction  of 
stroke-written  symbols  within  the  available  flyback  time.  The  limiting  factor 
is  the  quantity  of  written  symbology  required  during  the  one  millisecond  of 
vertical  retrace  time. 

The  dual- mode  deflection  system  is  capable  of  either  sequential  caligraphic 

beam  steering  as  for  the  wideband  deflection  system  in  purely  symbolic  modes, 

or  a more  efficient  scanning  technique  which  minimizes  power  dissipation  i 

during  the  TV  raster  scanning  mode.  This  design  couid  also  include  a limited 

raster  symbol  overlay  using  video  mixing  techniques  during  the  TV  raster 

mode.  All  system  performance  requirements  are  reduced,  since  the  stroke 

writing  applies  only  to  the  symbolic  mode  over  which  some  17  milliseconds 

(60  field  rate)  are  available  and  this  results  in  a reduced  writing  speed  and 

simpler  waveform  generation.  However,  additional  complexity  is  needed  to 

generate  the  limited  number  of  raster  symbols  which  are  in  the  TV  raster 

scan  mode. 

The  last  option  is  the  all-raster  energy  recovery  deflection  or  tuned  system. 

This  approach  generates  all  the  synthetic  symbology  in  the  raster  and  mixes 
it  with  the  E/O  video  when  imaging  sensors  arc  selected.  By  comparison, 
this  technique  is  the  least  complex  since  there  is  no  writing  speed  limitation 
on  symbol  generation.  The  TV  raster  approach  is  the  most  cost-effective 
from  the  overall  system  point  of  view,  a fact  proven  by  its  choice  for  use  in 
commercial  television  receivers.  The  kev  advantages  are:  1)  simple,  low- 
cost,  low-power,  highly  reliable  and  easily  maintainable  CRT  indicator  design; 

2)  simple  "in- raster"  symbol  generation;  3)  direct  compatibility  with  many 
electro-optical  sensors;  4)  a simple  interface  for  repeater  displays;  5)  com- 
patibility with  standard  TV  video  tape  recording  and  playback  equipment  for 
rapid  mission  evaluation  and  training;  and  6)  compatibility  with  TV  video  data 
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links  for  real-time  transmission  of  radar  video  for  evaluation  and  analysis  at 
remote  sites  equipped  for  TV  reception  and  displays.  The  disadvantage  of  the 
"in- raster"  symbol  writing  is  that,  in  some  areas,  the  symbol  becomes 
indistinguishable  unless  techniques  such  as  black  bordering  of  the  symbol  is 
used.  However,  in  this  case,  the  black  border  obliterates  some  of  the  image. 

Even  so,  raster  symbol  generation  is  generally  more  effective  for  display  with 
a large  quantity  of  characters,  while  the  cursive  approach  has  an  advantage 
for  displays  of  symbols  having  a large  quantity  of  vectors  or  circles.  The 
raster  approach  also  has  a greater  potential  foi  inverting  character  contrast, 
generating  electronic  windows,  drawing  broad  lines,  and  generating  other  area 
symbols.  If 
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Imaging  Combining  Techniques 


Rear-Port  CRTs  --  Image  combining  with  a rear -port  CRT  has  already  been 
discussed  for  moving  map  and  CRT  imagery.  This  device  has  a flat  optical 
window  near  the  gun,  which  permits  projecting  an  optical  image  on  the  back 
surface  of  the  phosphor. 

The  brightness  of  a rear-port  CRT  is  limited  by  the  projection  optics. 
Limited  space  within  a display  package  not  only  confines  the  heat  for  high 
power  (high  brightness  lamps)  operation,  also  tends  to  burn  the  film.  Good 
visibility  of  the  fine  detail  on  conventional  maps  and  charts  is  d iff i c i It  under 
high  ambient  illumination  conditions,  Iherefore,  a map  image  is  generally 
limited  to  low  brightnesses  of  between  100  and  200  loot- lamberts,  and  must 
be  viewed  with  a hood  in  direct,  bright  sunlight. 

Another  problem  with  optical  projection  systems  is  the  hot  spot  in  the  center 
of  the  screen.  This  is  generally  avoided  by  roughening  the  glass  on  the  in- 
side of  the  CRT  to  diffuse  the  image. 

Brightness  of  rear-port  CRT  phosphors  can  exceed  200  foot -lamberts,  but 
superimposed  map  imagery  would  then  not  be  visible.  Larger  display  pack- 
ages may  permit  brighter  map  images,  but  phosphor  brightness  is  reduced 
(one  half)  by  the  lack  of  an  aluminized  screen  behind  the  phosphor  (light 
from  the  projection  optics  cannot  penetrate  the  aluminum  film). 

Applying  a rear  port  to  a DVST  is  not  feasible  since  the  storage  mesh  is 
opaque.  This  suggests,  in  addition,  investigating  an  optically  transparent 
storage  mesh.  At  the  present  time,  rear-port  CRTs  are  limited  to  low- 
ambient  lighting  conditions,  use  with  a viewing  hood. 


Dual  Phosphor  CRTs  --  There  are  at  least  two  approi  s that  use  dual  re- 
sistance phosphor  CRTs  for  image  combining.  The  first  technique  uses  one 
gun  that  excites  both  a blue,  short -per sistance  phosphor,  and  an  orange, 
long-persistance  phosphor.  Blue  and  orange  filters  reduce  the  interference 
from  the  unwanted  phosphor  persistance.  The  blue  phosphor  filter  permits  a 
fast  raster  scan  TV  sensor  image  to  be  shown  while  the  orange  filter  permits 
viewing  the  slow-scan  radar  imagery.  This  combined  approach  penalizes 
both  modes  due  to  the  attenuation  of  the  filters.  Furthermore,  the  drive 
electronics  and  display  must  function  at  two  different  scan  rates.  A second 
approach  excites  the  phosphors  by  electron  beams  with  different  acceleration 
potentials.  The  lower  acceleration  beam  excites  the  short  persistence 
phosphor  alone  for  imagery  at  TV  rates.  The  higher  acceleration  potential 
excites  both  the  long-  and  short- persistence  phosphors  for  radar,  slow-scan 
imagery  alone.  This  technique  has  the  advantage  of  requiring  no  color  filters. 

The  dual-persistance  phosphor /filter  CRT  is  most  useful  where  no  ambient 
light  falls  on  the  display,  and  the  operator  has  little  time  pressure  and  no 
competing  task.  These  disadvantages  can  only  be  justified  where  cost  is  an 
overriding  consideration,  or  where  a radar  operator  is  a part  of  the  crew. 

The  dual  acceleration-potential  CRT  has  greater  potential  for  brightness,  and 
therefore  usefulness,  since  it  does  not  use  optical  filters.  However,  all 
approaches  using  long-persistance  phosphors  have  all  the  disadvantages  dis- 
cussed earlier:  low  brightness;  and  either  a rapid  fade  or  smearing.  Other 
approaches  would  probably  be  preferred. 

Multimode  Tonotron  (Hughes  Aircraft  Co.  ) --  The  multimode  Tonotron  tube, 
a special  type  of  a DVST,  is  discussed  as  a separate  image  combining  tech- 
nique since  it  has  a potential  for  sequentially  presenting  radar  and  TV  imagery. 
The  Tonotron  has  five  guns:  a stored  writing  gun,  a TV  gun,  a symbology  gun. 
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an  erase  gun,  and  a flood  gun.  The  stored  wiring  and  flood  guns  permit 
projection  of  radar  imagery  with  a high  viewing  brightness,  good  resolution, 
good  contrast  and  a good  gray  scale.  The  TV  gun  permits  using  the  standard 
short-persistence  phosphor  of  the  Tonotron  for  E-O  imagery  with  good  resolu- 
tion and  good  gray  scale,  but  with  brightness  too  low  for  viewing  under  high 
ambient  light  conditions.  Non-stored  symbology  is  presented  by  the  separate 
symbology  gun,  which  permits  symbology  to  be  moved  across  the  stored  video 
image  without  smearing. 

The  erase  gun  allows  selective  erasure  of  a stored  image.  Through  its  use, 
smearing  of  slow-scan  radar  imagery  is  avoided  by  erasing  the  previous 
image  sweep  before  writing  in  a new  sweep.  Consequently,  the  multimode 
Tonotron  can  present  acceptable  radar  imagery  for  target  recognition  with 
non-smearing  symbology.  However,  its  high-quality  TV  imagery  can  only  be 
viewed  under  low  ambient  light  level  conditions  (or  with  a hood).  Stored  radar 
and  TV  imagery  can  be  viewed  simultaneously  but  not  with  non-stored  symbols, 
unless  the  TV  and  symbol  gun  are  time-shared.  The  limitation  of  the  Tono- 
tron in  aircraft  may  be  its  sensitivity  to  stray  magnetic  fields  and  vibration. 

Video  Mixers  --  Many  systems  are  currently  available  for  mixing  TV- 
compatible  video  imagery.  These  systems  routinely  offer  as  many  as  100 
special  effects  pattern-generators.  They  include  wipe  patterns  (inserting  or 
transitions  between  video  channels)  such  as  bars,  rotary,  windshield,  and 
many  other  combinations.  Video  inputs  can  be  mixed  or  dissolved  into  each 
other  with  many  types  of  edge  and  border  effects. 

Mixing  is  accomplished  on  an  additive  basis  with  faders  controlling  the 
brightness  of  eacl  channel  with  an  attenuation  from  0 to  100  percent.  Wipe 
patterns  can  be  automatically  programmed  with  transitions  of  from  10  to  one- 
tenth  of  a second.  Black,  white,  or  drop  shadow  borders  can  be  created 
around  alpha-numerics.  Spotlight  effects  can  be  created  with  circular, 
rectangular  or  diamond  patterns  at  any  ratio  of  brightness  between  the  back- 
ground and  the  scene  within  the  spotlight.  Joystick  positioners  are  available 
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to  move  single-wipe  patterns  anywhere  within  the  display  field.  The  aspect 
ratio  of  the  special  effect  patterns  can  also  be  varied  by  a single  control. 
Polarity  can  be  inverted  and  blink  rates  varied. 


The  variety  of  special  effects,  ease  of  operation,  and  the  preprogrammed 
automatic  control  of  these  video  mixes  is  very  extensive,  and  indicates  a 
well-developed  arsenal  of  circuitry  and  technique.  Consequently,  many  forms 
of  video  inserting  can  easily  be  accomplished  for  aircraft  application,  with  a 
minimum  of  technical  risk.  The  major  developmental  task  is  to  convert  com- 
mercial units  to  militarized  hardware. 

Scan  Conversion  Tubes  --  The  problems  with  DVSTs  and  multimode  Tono- 
trons  (DVSTs)  have  led  to  a need  for  a device  to  store  slow-frame-rate 
imagery  separately.  Such  a device  (a  scan  converter)  can  provide  all  the 
advantages  of  a DVST,  as  well  as: 

• Superior  display  using  a CRT  TV  monitor 

• Large  selection  of  CRT  sizes  and  shapes 

• Potential  application  to  flat-panel  displays 

• Nonsmearing  symbology  by  "in  raster"  or  stroke  writing 
between  frames 

• Flexibility  and  growth  potential  to  new  sensors 

• Multiple  displays  with  one  scan  converter 


Scan  conversion  tubes  tend  to  fall  into  one  of  two  main  categories:  the  single- 
ended,  and  the  double-ended.  In  the  single-ended  version  the  charge  is  de- 
posited on  and  rend  from  the  same  side  of  a storage  mesh.  A single- 
deflection  system  is  used  which  is  shared  by  the  read  and  write  beams.  It 
follows  that  the  reading  and  writing  operation  must  be  time-multiplexed, 
which  means  that,  for  many  applications,  two  tubes  are  needed  for  a page-bv- 
page  mode. 
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The  double-ended  storage  tube  has  separate  read  and  write  beam  deflection 
systems  on  either  side  of  the  storage  mesh.  Charge  is  developed  on  one  side 
of  the  mesh  by  the  write  beam  and  interrogated  on  the  other  side  by  the  read 
beam.  The  charge  pattern  is  written  on  the  dielectric  mesh  by  either  a 
secondary  emission  or  a bombardment-induced  conductivity.  The  pattern 
on  the  dielectric  modulates  the  scanning  read  beam  on  the  output  side.  The 
read  beam  is  scanned  at  the  standard  TV  format  of  60  fields  per  second, 
while  the  write  input  beam  can  be  at  any  scan  rate  or  format. 


The  advantage  of  the  double-ended  scan  converter  tube  is  that  the  write-in 
and  read-out  operations  can  be  performed  simultaneously.  Its  disadvantage 
is  the  extreme  complexity  of  adjustment  required  during  installation  or  re- 
placement. Since  the  adjustments  tend  to  vary,  the  periodic  maintenance  costs 
are  also  high.  The  end  result  is  a system  which  is  inadequately  maintained, 
has  a high  replacement  cost  and  low  reliability,  and  has  poor  accuracy, 
dynamic  range,  and  picture  registration. 


These  problems  can  be  avoided  through  the  use  of  two  single-ended  scan  con- 
version tubes.  The  procedure  is  to  write  one  frame  of  radar  imagery  into  one 
of  the  tubes.  The  next  frame  of  radar  imagery  goes  to  the  second  tube.  The 
TV  monitor  is  switched  between  the  tubes  at  the  frame  rate  of  the  radar 
sensor.  The  image  seen  on  the  TV  display  is  frozen  over  the  radar  frame 
period,  approximately  one  second.  The  disadvantage  of  this  approach  is  that 
the  display  is  not  updated  continuously  at  the  sweep  rate  of  the  radar  (as  with 
the  double-ended  tube).  A further  disadvantage  is  that  the  tube  deflection 
circuitry  must  operate  at  both  the  slow-radar  and  fast-TV  raster  rates. 


This  approach  has  advantages 


Erase"  before  "write"  is  not  required 


Close  tolerance  mechanical  alignments  for  fine  correction  for 
deflection  gain  and  offset  are  not  required  when  replacing 
tube  assemblies 


Ease  of  packaging:  the  single-handed  tubes  are  less  than 
half  as  long  as  the  double-handed  tubes 
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• The  tube  and  coil  assembly  need  not  be  accurately  aligned 

• Unit  operation  does  not  depend  upon  the  position  of  the  two  tube 
assemblies  with  respect  to  each  other 

• Lower  cost 

Digital  Scan  Conversion  --  Storage  tube  scan  converters  can  successfully 
meet  specific  needs  of  unsophisticated  display  systems.  Until  quite  recently 
they  have  been  cheaper,  smaller,  and  consumed  less  power  than  digital  scan 
converters  of  identical  specifications.  The  main  advantage  of  the  storage 
tube  is  the  virtual  independence  of  its  read  and  write  scan  patterns.  Howeve 
it  is  difficult  to  scan  in  random  or  discontinuous  steps  because  of  the  self- 
inductance of  the  deflection  coils.  For  this  reason,  display  features  which 
require  variable  read-out,  frame  overlap  and  image  magnification  tend  to  be 
performed  easier  with  a digital  system.  Furthermore,  rolling  map  presents 
tions  of  moving  platform  radars  are  difficult  to  achieve  with  storage-tube 
converters.  Also,  the  ten  shades  of  gray  of  tonal  quality  of  a stored  picture 
is  below  that  of  commercial  TV.  The  high  impedance  of  the  device  lowers 
the  signal-to-noise  ratio,  which  degrades  the  overall  picture  quality.  This 
degradation  is  compounded  by  areas  of  uneven  leaks  which  produce  patchy 
images.  An  optimal  picture  requires  highly  stable  drive  and  deflection  wave 
forms  with  careful  setup  and  frequent  readjustments.  Filament  life  is  anoth 
problem.  Furthermore,  the  device  is  highly  sensitive  to  extraneous  magnet 
fields  since  the  electrons  in  the  read  beam  are  repelled  by  the  stored  charge 
and  the  velocity  of  the  electrons  is  momentarily  reduced  to  zero.  The 
environmental  shortcomings  of  the  storage  tube  seriously  limit  its  use  in  a 
miliary  vehicle. 
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These  limitations  have  led  to  the  development  of  digital  scan  conversion 
techniques,  A digital-scan  converter  must  be  differentiated  from  a solid- 
oiate  converter  to  avoid  ambiguity.  The  solid-state  converter  is  typified 
by  the  charge-coupled  devices  discussed  in  the  next  section.  The  digital- 
scan  converter  is  also  a solid-state  device,  but  relies  more  on  traditional 
digital  logic  design  than  the  solid-state  converter.  All  solid-state  systems 
are  reliable,  easily  maintained  and  require  little  or  no  readjustment  in  use. 
In  addition,  digital  systems  are  easier  to  partition  and  simpler  to  integrate 
with  existing  computer  systems.  The  main  advantage  of  a digital  system  is 
its  capability  to  produce  a picture  of  definable  quality  within  the  constraints 
imposed  by  storage  capacity,  volume,  and  power  consumption. 

The  digital-scan  converter  has  additional  capabilities  r 

• Picture  hold 

• Rolling  update  presentation 

• Rolling  map  presentation 

• Page-by-page  presentation 

• Variable  line-by-line  integration 

• Field  or  frame  update 

® Polarity  reversal 

• Reservable  line  count 

• Input  magnification 

• Image  super-imposition 

• Selective  erase 

• Controlled  fade  erase 

• Multi -input 

• Choice  of  TV  raster  rate 

• Contrast  bit  suppression 

• Reduction  of  resolution 

• Histogram  equalization 

• Spatial  filtering  and  gamma  correction 
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• Clutter  suppression  and  scan  correlation 

• Synthetic  digital  maps 

• Level  slicing 

• Scale  and  geometry  conversion 

A digital  scan  converter  samples  the  incoming  analog  video  and  converts  it 
to  digital  codes  corresponding  to  the  number  of  amplitude  levels  (shades  of 
gray)  desired  for  each  resolution  element  on  the  display.  The  digitized  video 
is  then  stored  in  digital  memory,  which  is  read  out  at  TV  rates  and  converted 
to  analog  video  for  input  to  a TV  display  monitor  (Figure  4-17).  It  is  neces- 
sary that  each  storage  cell  be  capable  of  encoding  enough  contrast  levels  to 
reproduce  the  image  to  the  desired  degree  of  fidelity.  In  practice,  the  maxi- 
mum gray-scale  resolution  is  limited  by  either  the  sensor,  the  display,  or 
the  visual  acuity  of  the  observer.  Figure  4-18  (Ref.  13)  shows  the  memory 
and  bandwidth  requirements  as  a function  of  resolution  based  upon  a 4 -bit 
(16  gray  scales)  digital  code  for  each  resolution  element.  The  video  band- 
width requirement  assumes  a 30-IIz  TV  frame  rate  with  a 75  percent  active 
display  duty  cycle. 

Selective  erasure  of  video  information  is  automatically  obtained  with  digital 
storage,  since  each  resolution  element  is  erased  as  new  sensor  data  are 
stored.  A digital  scan  converter  provides  high-quality  imagery  because: 
a)  the  digital  memory  is  perfectly  uniform;  b)  it  can  be  designed  to  store  as 
much  sensor  resolution  as  desired;  c)  it  stores  the  full  signal  amplitude  with 
as  many  shades  of  gray  as  desired;  d)  the  image  does  not  fade  as  a function 
of  time  or  environment,  and  3)  the  stored  signals  can  be  read  out  at  a high 
signal /noise  ratio. 

It  is  expected  that  digital-scan  conversion  will  find  w'ider  applications  in  new 
avionics  systems,  as  the  cost  of  digital  memories  and  components  continue 
to  decline.  Volume  and  cost  of  the  converter  is  related  to  the  computer 
memory  market.  Consequently,  the  volume  of  a digital-scan  converter  can 
be  reduced  to  one-quarter  and  the  cost  reduced  by  a factor  of  five  within  the 
next  five  years  (Ref,  14). 
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Figure  4-17,  Basic  Digital  Scan  Converter  Block  Diagram 


16  AMPLITUDE 
lEVliS  STORED 


Solid-State  Scan  Conversion  Using  Charge  Coupled  Devices  (CCD)  --  There 
are  at  least  two  techniques  that  use  CCDs  for  image  combining.  The  first 
stores  alternate  fields  to  permit  mixing  of  FLIR  and  LLLTV  sensors  at 
different  line  rates;  i.  e.  , for  this  example  525  and  875  TV  lines /frame.  The 
second  approach  uses  a CCD-imaging  sensor  to  convert  slow-scan  radar 
imagery  (on  a DVST)  to  TV-compatible  format.  The  advantages  over  digital 
or  doubled-ended  tube  scan  conversions  are  cost  and/or  simplicity  and 
volume. 

CCD  Electronic  Scan  Conversion  --  Current  CCD  technology  will  permit 
frame  storage  of  raster  imagery  to  convert  video  from  one  frame  rate  to 
another,  for  example  from  525  TV  lines /frame  to  875  TV  lines.  It  will  also 
permit  a non-interlace  to  interlace  format  conversion,  or  vice  versa.  Two 
CCDs  must  be  used  for  alternate  field  or  frame  storage  since  "write  in"  and 
"read  out"  are  sequential  operations  for  CCDs  (Figure  4-19). 

This  approach  implies  a need  for  the  simultaneous  image  presentation  of 
non-compatible  raster  FLIR  and  LLLTV  sensors.  If  sequential  presentation 
were  required,  it  would  be  simpler  to  use  displays  that  can  be  switched  to 
alternate  line  rates.  It  is  further  assumed  that  a 525-TV  line  T-'LIR  sensor 
and  a higher  resoRition  875-TV-line  LLLTV  sensor  with  equal  FOVs  are 
being  displayed  on  an  875-TV-line  monitor.  This  requirement  is  realistic 
since  high-resolution  LLLTV  875-TV-line  sensors  are  common,  while  high- 
resolution  FLIRs  are  not. 

The  Fairchild  CCD361  23,  560-bit  analog  shift  register  is  intended  for  analog 
signal  processing  and  temporary  storage  applications.  It  is  organized  in  a 
series -parallel-series  configuration.  Input  voltage  is  sampled  by  a charge 
injection  port  and  generates  proportional  charge  packets  in  a 190-element 
serial-input  register.  The  information  is  then  transferred  in  parallel  to 
registers  that  transport  the  packets  to  a 190-element  serial  output  register. 
Clocking  the  output  Register  then  delivers  the  charge  package  to  an  on-chip 
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Figure  4-19.  CCD  Electronic  Frame  Scan  Conversion  System 
for  Simultaneous  Presentation 


output  amplifier  to  be  converted  to  an  analog  voltage.  Continuous  clocking 
of  the  device  reproduces  a line-by-line  representation  of  the  190  input  ele- 
ments at  the  output. 

The  CCD361  can  be  used  for  a wide  variety  of  applications.  The  simplest 
application  is  for  delay  of  analog  information  which  is  accurately  controlled 
by  an  external  clock.  Varying  the  clock  simply  changes  the  delay.  One 
specific  delay  application  appropriate  to  its  large  capacity  is  that  of  a large 
number  of  analog  samples  for  video  frame  storage.  This  permits  a time 
axis  scan  conversion  where  write  and  read  rates  can  be  varied  to  adapt  to 
particular  display  requirements.  A line  of  analog  video  information  can  be 
read  in  and  stored  at  a low  line  rate  and  then  read  out  at  the  higher  line  rate. 
For  example,  490  active  lines  of  the  low  resolution  video  are  stored  to 
match  the  525  line  input  system  rate  (see  Figure  4-19).  A logic  circuit  is 
required  in  the  video  mixer  (output  of  CCDs,  see  Figure  4-19)  to  feed  817 
active  lines  to  the  875  TV  display  from  the  stored  490  active  TV  lines.  This 
logic  circuit  will  be  required  to  repeat  327  lines  of  the  490  stored  lines. 
Therefore,  one  line  will  be  repeated  on  the  average  of  every  1.  5 stored  ac- 
tive line.  This  will  result  in  a slight  distortion.  Also  the  LLLTV  sensor 
will  have  a better  resolution  than  the  FLIR  sensor. 

"Write  in"  clock  rates  are  generated  from  the  FLIR  sensor  input  sync  pulses 
and  the  clock  rates  for  CCD  "read  out"  is  generated  by  the  875  TV  line  FLIR 
sensor  sync  pulses.  The  "A"  CCD  field  memory  is  read  out  in  near  reel 
time,  while  the  "B"  CCD  field  memory  is  in  the  "write  in"  mode  (see  Fn 
4-19).  "Read  out"  and  "write  in"  modes  are  alternated  from  field  to  • 
between  "A"  and  "B"  CCD  memory  units, 

A block  diagram  of  the  system  which  stores  a single  field  > - ■ 

Figure  4-20.  The  input  demultiplexer  (DEMUX-1)  recent  • 
data.  The  first  124  lines  of  video  are  fed  to  DEMUX-2,  ile  ••  < 
lines  are  fed  to  DEMUX-3  (a  total  of  245  active  lint 
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Each  of  the  124  lines  is  divided  into  thirds  by  the  demultiplexer,  the  first 
190  points  in  the  line  are  routed  to  the  first  analog  shift  register,  the  second 
190  to  the  second  register  and  the  third  190  to  the  third  register  (a  minimum 
of  460  horizontal  resolution  elements  are  needed  to  match  the  vertical  resolu- 
tion of  350;  i.  e.  0,  7 Kell  factor  x 490  active  lines  x 4 :3  aspect  ratio). 

After  the  first  124  lines  are  stored  in  registers  1,  2,  and  3,  the  next  124 
lines  are  stored  in  registers  4,  5,  and  6. 

However,  full-frame  (two  fields)  storage  is  necessary.  This  requires  that 
two  field  systems  be  combined  and  an  adtu  -1  demultiplexer  and  multiplexer 
be  added. 

A block  diagram  of  the  full-frame  storage  system  is  shown  in  Figure  4-21. 

The  Field  1 and  Field  2 systems  referenced  here  are  identical,  and  each  is 
equivalent  to  Figure  4-20.  The  input  demultiplexer  routes  the  odd  fields  to 
the  Field  1 system  and  the  even  fields  to  the  Field  2 system. 

Another  approach  to  combining  non-compatible  raster  imagery  is  possible 
when  a smaller  image  is  permissible  for  one  sensor  than  for  the  other.  If 
the  490  active  lines  of  a 525-line  raster  are  stored  in  a solid-state  scan  con- 
verter, we  can  clock  them  out  directly  on  the  490  lines  located  in  the  center 
of  the  875-line  display.  To  do  this  a vertical  blanking  signal  added  to  the 
storage  output  would  simply  be  extended  to  cause  the  picture  to  begin  well 
below  the  top  of  the  875-line  display,  is  shown  in  Figure  4-22.  By  shortening 
the  picture  vertically,  we  have  distorted  it.  The  true  proportions  of  the  image 
can  be  restored  by  also  narrowing  the  image  by  increasing  the  clock  rate  and 
inserting  extended  horizontal  blanking  signals.  The  picture  now  appears  only 
in  the  center  of  the  875-line  raster  (Figure  4-22).  The  image  is  correct  in 
proportion  and  contains  all  the  original  picture  information. 

The  composite  video  signal  for  the  image  in  Figure  4-22  is  compatible  in 
every  way  with  other  875-line  video,  and  can  be  mixed  with  the  output  of  an 
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Figure  4-21.  Full-Frame  Storage  System,  30  Frames  per  Second 
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Figure  4-22.  Simplified  Solid-State  Scan  Conversion 
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875-line  sensor  as  shown  in  Figure  4-22.  A dual  CCD  solid  state  converter 
is  used  to  store  the  output  of  the  525-line  FLIR.  The  readout,  however,  is 
synchronized  with  the  blanking  signals  generating  video  only  in  the  center  of 
the  87  5-line  format.  Figure  4-22  compares  the  resulting  video  waveforms 
with  those  from  a more  traditional  scan  converter. 

The  FLIR  imagery  will  appear  on  the  display  as  shown  in  Figure  4-22.  When 
mixed  with  the  LLLTV  imagery  it  will  cover  only  the  central  portion  of  the 
TV  image,  leaving  the  edges  unaffected.  The  scaling  of  the  two  images 
determines  the  fields  of  view  of  the  optics  used  on  the  two  sensors. 


atical  Scan  Conversion  — 


CCD  Sensor  --  The  Fairchild  CCD-221A  solid-state  imaging  detector  is 
a fully  TV-compatible  380-element  x 488-element  sensor  device.  A camera 
using  the  CCD-22 1A  sensor  can  be  developed  to  generate  TV-compatible  video 
from  slow-scan  radar  imagery  presented  on  a DVST,  as  shown  in  Figure  4-23. 
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This  application  of  a CCD  camera  for  radar  imagery  scan  conversion  assumes 
space  is  available  to  mount  the  DVST  and  camera  at  a site  remote  from  the 
display.  The  overall  length  of  one  approach  to  CCD  camera  scan  conversion 
is  17.  75  inches,  which  assumes  an  eight-inch  long  H-1126AP20  Tonotron,  a 
six-inch  optical  object  distance  and  a 3.  75-inch  camera. 


The  advantages  of  a CCD-22 1A  solid-state  sensor  over  a double-ended  scan 
converter  are  ruggedness,  reliability,  longevity,  low  voltage,  low  power  re 
quirements,  a small  size  and  a light  weight. 


General  Description  — The  CCD221  is  a solid-state,  self-scanned  area 
image  sensor  suitable  for  use  as  the  sensor  in  camera-type  applications.  The 
device  is  organized  in  an  array  of  488  horizontal  lines  by  380  vertical  columns. 
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Figure  4-23.  Radar  Scan  Conversion 


The  185,  440-image  sensing  elements  are  14  (im  horizontally  by  18  pm  vertical- 
ly, and  are  located  on  30  pm  horizontal  centers  and  18  pm  vertical  centers. 

The  dimensions  of  the  image  sensing  area  are  8.  8 mm  x 11.4  mm,  with  a 
diagonal  dimension  of  14.4  mm. 

The  X-Y  format  of  the  array  was  selected  to  provide  a 4 x 3 image  aspect 
ratio  and  to  approximate  the  image  size  of  standard  NTSC  systems.  The 
highly  precise  location  of  the  photosites  allows  an  accurate  identification  of 
each  component  of  the  image  signal.  This  feature  allows  the  device  to  be 
used  in  applications  requiring  dimensional  measurement.  This  sensor  is  in- 
tended to  be  used  in  video  cameras  that  require  a low  power,  a small  size, 
high  sensitivity  and  high  reliability.  It  is  enclosed  in  a 36-pin  dual,  in-line 
package  with  an  optical  glass  window: 

• 185,440  image  sensing  elements  on  a single  chip 

• Column  anti-blooming 

• No  lag,  no  geometric  distortion 

• Gamma  characteristic  of  approximately  1.  0 

• On-chip  video  preamplifier  providing  more  than  200mV  of 
output  signal 

• High  dynamic  range:  Typically  300:1 

• Low  light  level  capability;  low  noise  equivalent  exposure 

• Wide  range  of  video  data  rates,  up  to  75  frames/sec 

• All  operating  voltage  under  20V 

• Low  power  dissipation:  Typically  75mW 

• High  reliability 


Absolute  Maximum  Ratings  — 

Operating  Temperature:  -25°C  to  55°C 

Storage  Temperature:  -25°C  to  100°C. 

Voltages  -- 

10V  to  +15V 

Functional  Description  — Light  energy  incident  on  the  image  sensor  ele- 
ments generates  a packet  of  electrons  at  ea^h  sensing  element.  Electrical 
clocking  of  the  photogate,  the  vertical  analog  txansport  registers,  and  the 
horizontal  analog  output  register  sequentially  delivers  the  charge  packets  to 
the  preamplifier. 

Image  Sensing  Elements  — Image  photons  pass  through  a transparent 
polycrystalline  silicon  gate  structure  and  create  hole-electron  pairs  in  a 
single  crystal  silicon  layer.  The  resulting  photoelectrons  are  collected  in 
photosites  during  the  HIGH  state  of  the  photogate.  The  amount  of  charge  ac- 
cumulated is  a linear  function  of  the  incident  illumination  intensity  and  of  the 
integration  period. 

Vertical  Analog  Transport  Registers  --  Charge  packets  are  transferred 
out  of  the  array  in  two  sequential  fields  of  244  lines  each  at  the  end  of  an 
integration  period.  Alternate  lines  of  charge  packets  are  transferred  to  their 
corresponding  sites  in  the  vertical  registers  when  the  photogate  voltage  is 
lowered.  Clocking  of  the  vertical  register  (two-phase)  delivers  the  charge 
packets  from  the  380  vertical  registers  to  the  horizontal  analog  transport 
register.  A minimum  of  246  vertical  transfers  (24  6 clock  cycles)  are  re- 
quired to  transport  each  field  of  charge  packet  out  of  the  vertical  registers. 

A second  field  cycle  is  initiated  to  receive  the  information  from  photosites 
corresponding  to  the  other  field  (i.  e.  , the  even-numbered  photoelements). 
Clocking  of  the  vertical  register  then  transports  the  charge  packets  to  the 
output. 
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Horizontal  Analog  Transport  Register  — The  two-phase  horizontal 
register  has  392  elements  and  receives  the  charge  packets  from  the  vertical 
transport  registers,  line  by  line.  Each  row  of  information,  received  from 
the  vertical  registers,  is  serially  moved  to  the  outpuc  amplifier  by  two  hori- 
zontal clocks. 

A minimum  of  392  horizontal  clock  pulses  are  required  to  completely  transfer 
one  row  of  information  to  the  floating  gate  amplifier. 

Floating  Gate  Amplifier  --  Charge  packets  from  the  horizontal  register 
are  sensed  by  a floating  gate  amplifier.  Its  output  potential  is  linear  to  the 
input  signal  charge  and  drives  a MOS  transistor.  The  transistor  output  then 
drives  the  gate  of  an  output  n-channel  MOS  transistor  to  produce  a video  out- 
put signal. 

Optical  Combining  Techniques  --  Optics  can  also  be  used  to  combine  informa- 
tion from  different  sensors.  This  approach  avoids  the  electronic  problems 
of  mixing  incompatible  E-O  sensor  videos  and  allows  a choice  of  display  tubes 
to  match  specific  sensor  designs  (Figure  4-24).  The  method  has  several  o+her 
advantages.  The  images  do  not  necessarily  appear  at  exactly  the  same  dis- 
tance from  the  observer;  this  can  make  the  job  of  discrimination  easier. 
Maintenance  of  the  individual  components  can  be  simplified.  Finally,  failure 
of  one  tube  or  electrical  part  will  not  result  in  the  loss  of  the  entire  display. 

Figure  4-2  5 shows  a flat  combiner.  The  coating  material  can  be  selected  to 
allow  the  proper  ratio  between  reflected  light  and  transmitted  light  (Beam  2). 
This  is  a design  tool  which  can  be  used  to  balance  the  intensity  of  the  display. 
The  coating  material  can  also  be  designed,  within  limits,  to  selectively  re- 
flect certain  wavelengths  of  light.  Combining  this  with  a proper  tube  phosphor 
can  also  enhance  the  light  balance.  It  is  standard  practice  to  use  anti- 
reflective  coatings  on  other  surfaces  of  combiners  to  eliminate  ghost  images. 
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Figure  4-25.  Flat-Plate  Combiner 
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Lenses  — Lenses  can  be  used  to  increase  the  size  of  the  image  and  1o 
transfer  the  image  forward,  near  the  external  opening  of  the  display.  A 
field  lens  can  be  used  to  assure  that  light  from  the  display  enters  the  exil 
pupil.  Specific  lens  systems  for  each  of  these  tasks  can  only  be  designed  for 
specific  applications.  The  following  does  not  attempt  to  discuss  the  best  lens 
system  design  for  some  display,  but  rather  to  provide  a "feel"  for  what  can 
be  done  with  lenses  and  for  what  their  use  implies  in  terms  of  display  size, 
limitations  of  eye  movement  and  other  operational  questions. 

Transfer  Lenses  --  In  the  simple  combiner  design  (Figure  4-27),  the 
image  seen  by  the  pilot  would  more  desirably  be  at  the  front  surface  of  the 
display.  This  can  be  done  using  a transfer  lense  which  can  also  magnify  the 
image.  For  instance,  if  the  image  size  desired  were  6 inches,  the  ray  trace 
shown  in  Figure  4-29  would  result.  The  CRT  and  DVST  face  plates  are  10 
inches  from  the  exit  aperture  of  the  display.  To  magnify  the  four-inch  CRT 
image  to  six  inches  and  to  transfer  the  image  to  the  exit  aperture  requires 
a lens  with  a focal  length  equal  to  2.  4 inches.  However,  to  see  the  edge  of 
the  field  of  view  at  the  exit  pupil  20  inches  away,  the  lens  diameter  required 
is  eight  inches.  (This  is  shown  by  the  ray  label  "A-B"  in  the  figure.  ) The 
ratio  of  the  focal  length  to  the  lens  diameter  is  the  F-number  of  the  lens. 

It  is  practically  impossible  to  fabricate  a lens  faster  (lower  number)  than 
F/0.  5,  yet  the  lens  shown  would  have  to  be  F/0.  3.  This  is  clearly  impossible 
and  requires  something  be  done.  The  usual  solution  is  to  use  a field  lens. 

Field  Lens  --  Figure  4-30  shows  the  addition  of  a field  lens  to  this  de- 
sign. Its  optical  characteristics  are  chosen  to  re-image  the  transfer  lens  at 
the  exit  pupil.  In  this  way,  the  light  which  passes  through  the  transfer  lens 
is  assured  of  entering  the  exit  pupil,  but  the  magnification  of  the  system  is 
unchanged.  In  this  case,  the  field  lens  eases  considerably  the  optical  design 
of  the  transfer  lens.  It  allows  the  lens  diameter  to  reduce  to  about  two  inches. 
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yielding  F/l.  2.  However,  the  field  lens  focal  length  is  only  4.  62  inches  and, 
since  the  image  size  is  six  inches,  the  field  lens  itself  is  F/0.  77.  This  is 
possible  but  still  very  difficult.  Such  a system  could  be  constructed  as  shown 
in  Figure  4-31,  and  would  need  a volume  of  6 x 12  x 18  inches.  The  cube 
combiner  has  been  shown  modified  to  provide  for  a front -illuminated  screen 
projection  moving  map  display. 

APERTURE  DIA  = 8 IN. 

F = 2.40 

F/.30 


6 IN.  DIA  IMAGE 


The  main  reason  the  lens  requirements  are  so  tough  in  this  design  is  the 
short  distance  allowed  for  the  image  transfer  from  CRT  to  display  face. 

This  distance  is  lengthened  considerably  in  the  design  shown  in  Figure  4-32. 
Here  the  depth  and  the  height  of  the  display  are  kept  the  same,  but  the  width 
is  increased.  The  throw  distance  between  tube  and  image  is  increased  to 
32  inches  by  turning  the  display  tubes  around.  This  results  in  a transfer 
lens  focal  length  of  7.  68  inches  and  a field  lens  focal  length  of  9.  8 inches, 
F/3.  8,  and  F/l.  6 respectively.  There  is  also  much  more  room  for  the 
moving-map  display. 

This  system  seems  quite  practical  and  gives  a good  feel  for  the  size  neces- 
sary to  provide  a display  with  a six-inch  image. 


Compatible  FLIR  and  LLLTV 

Techniques  for  combining  imagery  from  compatible  FLIR  and  LLLTV 
sensors  can  be  divided  into  the  need  for  sequential  and  simultaneous  presenta- 
tions. Sequential  presentations  of  sensor  imagery  can  most  easily  be  satis- 
fied by  switching  the  display  to  the  desired  sensor  to  be  imaged  (Figure  4-33). 
Simultaneous  imaging  is  more  complicated  and  requires  frame-to-frame 
alternation  of  images,  a video  mixer,  a dual-beam  CRT,  or  optical  mixing 
techniques.  Image  frame-to-frame  alternation  requires  combining  by  the 
observer's  visual  response.  Each  image  now  has  a frame  rate  of  l5  Hz,  but 
the  overall  rate  is  still  60Hz.  Flicker  efforts  are  unknown  and  need  to  be 
investigated.  Video  mixers  (Figure  4-33)  are  commercially  available  and 
are  the  most  versatile  and  sophisticated  of  all  techniques,  out  are  limited  to 
video  signals  with  the  same  line  rate. 
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REFLECTING 


Figure  4-32.  Improved  Field  Lens  Design 
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Optical  mixing  (Figure  4-33)  is  certainly  a possibility  for  image  combining. 
However,  this  technique  would  require  two  CRTs  as  well  as  the  separate 
drive  circuitry.  There  does  not  appear  to  be  a distinct  advantage  to  this 
approach  over  the  video  mixer  or  over  the  dual-beam  technique. 


Non-Compatible  FLIR  and  LLLTV 

It  is  assumed  for  this  class  of  image  combining  techniques  that  the  FLIR 
sensors  have  a TV  raster  video,  but  that  the  line  scan  rates  are  not  the  same 
For  example,  high-resolution,  875-line  LLLTV  sensors  are  fairly  common. 
However,  FLIR  sensors  of  such  a resolution  are  beyond  the  general  state  of 
the  art.  Therefore,  the  presentation  of  high-resolution  LLLTV  and  low- 
resolution  FLIR  imagery  on  a single,  high-resolution  display  may  be  a fairly 
common  occurrence.  Again,  the  techniques  may  be  separated  into  the  need 
for  sequential  and  simultaneous  presentations. 

Sequential  presentations  of  sensor  imagery  can  best  be  accomplished  by  em- 
ploying a display  that  operates  at  either  sensor  scan  rate  (Figure  4-34). 

These  TV  monitors  are  available.  Three  options  are  possible  for  the  simul- 
taneous presentation  of  non -compatible  FLIR  and  LLLTV  sensor  images. 
These  are  alternate  frame  switching  on  the  same  display,  CCD  memory, 
and  optical  combining  techniques  (two  separate  displays,  see  previous 
section).  Simultaneous  presentations  by  alternate  frame  switching  on  one 
display  requires  the  monitor  to  operate  alternately  at  15  Hz  for  52  5 to  875 
line  rates. 

Figure  4-34  shows  the  application  of  two  Fairchild  CCD-361  analog  shift 
registers  to  provide  electronic  scan  conversion  of  the  52  5-line  FLIR  sensor 
to  an  875-line  rate  (see  earlier  section  for  a detailed  discussion).  This 
technique  would  require  12  Fairchild  CCDs,  seven  demultiplexes  and  seven 
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SENSOR 


a)  SEQUENTIAL  PRESENTATION  BY  DISPLAY 
SWITCHING  AT  DIFFERENT  RATES 


b)  SIMULTANEOUS  INSERT  WITH  CCD 
ELECTRONIC  SCAN  CONVERSION 


Figure  4-34.  Non- Compatible  FLIR  and  LLLTV  Image 
Combining  Techniques 
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multiplexers.  In  addition,  logic  would  be  required  to  address  the  multi- 
plexer and  the  demultiplexers.  The  advantages  of  this  form  of  scan  conver- 
sion over  scan  converter  tubes  are  volume,  cost  and  simplicity.  The  output 
of  the  CCU  electronic  scan  conversion  would  feed  into  a video  mixer,  as 
discussed  in  the  previous  section,  for  various  forms  of  in-setting,  fading  and 
so  forth. 

Raster  and  Radar  Image  Combining 

One  scan  conversion  device  and  two  techniques  for  combining  non-compatible 
TV  raster  and  radar  images  will  be  discussed  in  this  section:  CCD  cameras; 
optical  image  combining;  and  dual-persistence  CRT. 

The  CCD-221  solid-state,  self-scanning  sensor  has  the  potential  for  scan 
conversion  of  radar  imagery  presented  on  a DVST  (Figure  4-3  5).  The  resolu- 
tion capability  of  the  CCD  camera  is  approximately  equal  to  that  of  the  radar 
sensor  and  DVST,  and  there  will  be  some  loss  of  image  quality.  A high- 
resolution  vidicon  camera  would  provide  a better  image  quality,  but  this 
device  is  larger  and  less  reliable.  The  advantages  of  the  CCD  camera  are 
that  it  is  small.and  low  in  power  and  voltage  requirements. 

The  CCD  camera  is  a 52 5 -line  compatible,  and  its  output  then  can  be  fed 
directly  to  a video  mixer  with  a 52  5-line  E-O  sensor.  However,  if  the  E-O 
sensor  operates  at  an  875-line  rate,  then  both  the  video  mixer  and  camera 
must  operate  at  that  rate.  Video  mixers  at  87  5-lines  are  available  and  a 
scan  converter  is  required  to  obtain  875-line  rate  video  from  the  CCD  camera. 
The  CCD  camera  has  a high  sensitivity,  which  permits  use  of  a slow  lens  as 
well  as  a reduced  brightness  on  the  DVST.  This,  in  turn,  permits  a smaller 
spot  size  and  improves  resolution.  These  improvements  partially  overcome 
the  image  quality  loss  of  a CCD  camera.  The  approach  requires  space  for 
remotely  mounting  the  DVST  and  CCD  camera. 
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Figure  4-35. 


Non-Compatible  TV  Roster  and  Radar 
Image  Combining  Techniques 
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The  second  approach  to  the  simultaneous  inserting  of  non-compatible  E-O 
and  radar  sensor  imagery  is  optically  combining  (see  earlier  section)  with 
two  displays,  a DVST  and  a TV  monitor. 


A dual  persistence  phosphor  CRT  is  the  last  approach  < onsidered  for  com- 
bining radar  and  TV  imagery  (see  Figure  4-35  and  previous  dual  phosphor 
CRT  discussion). 

Combining  Symbology  and  Raster  Imagery 

Tradeoffs  for  combining  symbology  and  raster  imagery  have  been  discussed 
(see  previous  discussions  in  Section  IV). 

Raster  and  Projected  Imagery 

Figure  4-36  shows  one  application  of  a rear  port  CRT  tor  combining  radar 
imagery  with  optical  map  imagery.  The  radar  video  is  converted  to  a TV- 
raster  format  by  a scan  converter.  Radar  and  E-O  video  can  be  combined 
by  a video  mixer  if  they  are  at  the  same  TV  line  rates.  An  alternate  ap- 
proach to  mixing  map  imagery  with  an  optical  combiner  rather  than  a rear- 
port  CRT  is  shown  in  Figure  4-36  (see  earlier  discussions  of  optical  com- 
bining techniques). 
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-36.  Roster  and  Optical  Projected  Imagery 
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SECTION  V 
EVALUATION 


Table  5-1  provides  the  methodology  for  evaluating  candidate  image-combining 
techniques  discussed  in  Section  IV.  Evaluation  starts  with  how  well  hardware 
implementation  (left.  Table  5-1)  satisfies  the  weighted  display  characteristics 
(top,  Table  5-1)  as  shown  in  Table  5-1A.  Display  characteristics  are  weighed 
in  relative  order  of  importance,  with  the  most  important  at  the  top. 

The  second  step  of  the  evaluation  is  concerned  with  the  capability  (Table  5- IB) 
of  each  combining  approach  (implementation)  for  generating  each  method  of 
presenting  multiple  images  (bottom.  Table  5-1).  The  evaluation  (Table  5-1C) 
of  how  well  each  method  meets  the  needs  of  various  uses  for  multiple  imagery 
(right.  Table  5-1)  was  discussed  earlier  in  Section  III  and  is  repeated  for 
completeness. 

- 

Table  5- IB  shows  one  combining  approach  can  be  eliminated  from  further 
consideration  at  the  onset  and  another  can  be  reduced  in  importance.  The 
dual  persistence,  dual  acceleration  potential  CRT  is  satisfactory  only  for  slow 
juxtaposition  (alternation)  of  radar  and  TV  imagery.  Even  in  this  application, 
its  long- persistence  phosphor  has  poor  brightness  (Table  5-1A). 

A second  implementation,  optical  combining , can  only  combine  by  super- 
imposition  and  juxtaposition  when  used  alone  (Table  5-lB).  Split  screen  and 
insetting  are  difficult  optically,  however,  the  technique  is  not  rejected  since 
it  is  useful  when  used  in  concert  with  other  implementation  schemes. 

Since  all  other  implementations  satisfy  the  needs  of  combining  methods  equally 
well,  further  evaluation  must  be  based  upon  the  weighted  system  character- 
istics (performance  characteristic  rating.  Table  5-1A).  In  Table  5-2.  the 
potential  implementations  are  ranked  in  descending  order  of  desirability, 
defined  by  this  evaluation. 
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Table  5-1.  Evaluation  and  Ratings  of  Combining  Implementations 
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X - t naeceptablo 


NOTES  (Table  C): 

M - Most  promising 
P - Possible 
X - Inappropriate 
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Table  5-2.  Image  Combining  Implementation  Evaluation 
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Dual  Potential,  Dual  Not  acceptable. 

Persistence  CRT 
(Radar  and  TV 
imagery) 


The  multi-beam  CRT  and  image-alternation  implementations  have  the  highest 
ranking  (Table  5-2)  of  all  approaches  where  the  application  is  limited  to  TV 
line  rate  video  inputs  (requiring  only  short- persistence  phosphors).  Digital 
scan  conversion  is  the  best  overall  approach,  but  carries  complexity  and  cost 
penalties.  Optical  combining  implementation  has  the  most  promise  when 
combined  with  other  existing  techniques,  such  as  video  mixers,  map  projec- 
tors and  DVST.  However,  more  design  and  system  integration  study  is  needed 
to  solve  packaging  problems  which  now  make  optical  combinations  more  bulky 
than  those  using  electronic  combining  techniques. 

SYMBOLOGY  GENERATION 

Symbol  generation  by  either  in-raster  or  direct-draw  techniques  may  be 
required  to  implement  multifunction  image  combining.  When  symbology  is 
required,  the  type  to  be  used  must  be  selected  by  comparing  the  charac- 
teristics and  cost  of  the  available  generation  schemes  with  the  characteristics 
and  requirements  of  the  display  system.  In  general,  in-raster  generation  will 
be  more  versatile  and  less  costly  when  the  display  system  is  based  on  raster 
scanned  imagery  and  requires  alphanumeri cs  and  simple  line  symbols.  Where 
more  complex  or  higher  resolution  symbols  are  required,  direct  draw  during 
the  vertical  fly-back  period  of  the  raster  will  be  preferable,  though  it  will 
generally  be  more  expensive.  Tradeoffs  among  symbol  and  display  imple- 
mentations are  discussed  in  detail  in  Section  IV. 

SCAN  CONVERSION  TECHNIQUES 

Digital  scan  conversion  is  truly  an  implementation  of  image  combining  since 
two  video  signals  can  be  coded  digitally  and  stored  separately  in  memory 
relative  to  spatial  location.  The  stored  images  can  then  be  summed  before 
readout  to  build  up  a combined  image.  CCD  solid  state  and  tube  scan 


llfi 


conversion  devices,  by  themselves,  are  not  complete  image^ombining 
techniques  since  they  only  convert  noncompatible  video  to  a compatible 
format.  They  are  tools,  however,  which  can  be  used  in  conjunction  with 
combining  implementation  to  extend  the  capability  of  a multi- image  presen- 
tation. 

Table  5-3  presents  a ranking  of  how  well  each  conversion  technique  satisfies 
the  weighted  display  characteristics  introduced  in  Table  5-1. 


CCD  electrical  scan  conversion  has  the  best  ranking  for  converting  between 
TV  line  rates.  Its  disadvantage  may  be  the  initial  fabrication  and  wiring  of 
the  12  CCD  memory  chips.  However,  single,  large  memory  chips  will  be 
available  in  the  future.  Solid-state  CCD  optical  and  single-ended  tube  con- 
version have  about  the  same  ranking.  If  slow-scan,  low- resolution  imagery 
at  a 525  TV  line  output  rate  is  needed,  then  the  CCD  approach  is  best. 
However,  if  slow-scan,  high- resolution  imagery  at  an  875  TV  line  rate 
(or  higher)  is  needed,  then  single-ended  conversion  tubes  are  required.  If 
careful  design  techniques  are  not  followed,  these  devices  may  exact  a per- 
formance penalty  due  to  a poor  signal- to-noise  ratio  (SNR)  and  poor  dynamic 
range.  Double-ended  scan  conversion  tubes  are  unacceptable  due  to  their 
poor  accuracy,  high  complexity  and  resulting  poor  maintainability. 


Table  5-3.  Evaluation  of  Scan  Conversion  Techniques 
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SECTION  VI 
RECOMMENDATIONS 


To  supplement  these  recommendations,  the  main  conclusions  are  shown  in 
tabular  form  in  Section  V,  Table  5-1. 
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METHODS  AND  USES 

Four  uses  of  multiple  images  and  eight  methods  of  presenting  them  were 
identified.  Of  these  32  combinations,  10  appeared  to  be  inappropriate. 

Eleven  of  the  remainder  seem  promising,  while  the  rest  have  some  possi- 
bilities. All  seven  methods  involving  a single  screen  were  promising  for 
at  least  one  use.  However,  experimental  investigations  are  required  before 
a definitive  selection  of  methods  can  be  made  for  each  use. 

We  recommend  the  following  approach  to  research  in  this  area.  First,  the 
Navy  should  assign  priorities  to  the  four  uses  of  multiple  image  presentation. 
Those  uses  of  concern  should  be  investigated,  beginning  with  that  judged  to 
be  most  important. 

Appropriate  imagery  would  be  selected,  or  developed,  as  necessary.  The 
candidate  presentation  methods  would  be  simulated.  Experiments  would  be 
conducted.  In  these,  the  various  presentation  methods  under  investigation 
would  be  used  to  show  the  selected  imagery  to  observers.  The  response  of 
the  observers  to  this  imagery  would  be  recorded.  The  response  data  would 
be  analyzed  and  used  to  compare  the  relative  merits  of  the  presentation 
methods.  The  method  associated  with  the  most  effective  performance  would 
be  recommended. 

This  procedure  should  be  repeated  for  each  use  that  the  Navy  judges  to  be 
important. 
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[f  the  information  composite  use  were  selected,  several  types  of  image  con- 
tent relationship  should  be  used.  In  one  case,  the  two  images  from  different 
sensors  should  have  an  identical  pointing  angle,  scale  and  geometry.  In  addi- 
tion, imagery  should  be  used  in  which  pointing  angle,  scale  and  geometry  are 
varied  in  such  a way  as  to  provide  partially  related  imagery.  Completely 
unrelated  imagery  would  be  of  no  interest  in  this  study. 

If  one  investigated  the  composite  image  use,  similar  imagery  content  rela- 
tionships should  be  used.  In  addition,  the  imagery  sample  used  should  include 
examples  of  overlaying,  spotlighting  and  replacement  (as  discussed  in  Sec- 
tion III). 

Different  imagery  would  be  required  for  investigating  the  third  use.  When 
parallel  information  sources  are  monitored,  the  two  images,  being  viewed 
are  likely  to  have  little  or  no  relationship  in  terms  of  content.  However,  a 
number  of  image  pairs  should  be  used.  For  example,  LLLTV  and  FLIR, 
LLLTV  and  radar,  radar  and  map  imagery,  and  FLIR  and  symbolic  infor- 
mation. This  list  is  illustrative,  not  exhaustive. 

It  would  be  appropriate  to  investigate  methods  of  making  transitions  in  the 
context  of  target  acquisition  or  reconnaissance.  The  imagery  used  and  the 
instructions  to  the  observers  would  maximize  the  need  for  transitions. 

Changes  between  various  image  sources  should  be  investigated;  for  example, 
from  map  to  radar,  from  radar  to  LLLTV  and  from  LLLTV  to  FLIR. 

A general  approach  to  research  in  this  area  is  summarized  below: 

Task  1.  Prioritize  the  four  uses  of  multiple-image  presentation. 

Task  2.  Design  experiments  that  investigate  the  use  of  highest 
priority. 


120 


Task  3. 
Task  4. 

Task  5. 

Task  6. 

Task  7. 


Prepare  suitable  imagery. 

Construct  an  apparatus  capable  of  producing  the 
appropriate  methods  of  multiple-image  presentation. 

Conduct  experiments  to  measure  the  performance  of 
human  observers  for  each  presentation  method. 

Analyze  data,  comparing  performance  for  each  pre- 
sentation method. 

Report  findings,  recommend  best  method(s)  of  pre- 
sentation for  particular  use  investigated. 


Research  should  be  conducted  on  all  of  the  uses  deemed  to  be  important. 
This  is  the  only  way  to  produce  definitive  answers  linking  methods  to  uses 
of  multiple -image  presentation. 


IMPLEMENTATION 

We  identified  seven  different  methods  of  presenting  multiple  images  on  a 
single  screen.  Eight  techniques  or  devices  were  located  that  could  be  used 
in  implementing  these  methods.  The  characteristics  of  all  eight  were 
investigated  and  evaluated.  To  carry  this  analysis  further  and  to  obtain 
more  specific,  applied  data, it  will  be  necessary  to  relate  potential  systems 
to  particular  aircraft,  sensors  and  missions. 

We  recommend  the  following  program  to  generate  more  specific  data. 
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Task  1 


Either  choose  a specific  aircraft  with  specific  missions,  or  develop  a hypo- 
thetical model  of  an  aircraft  with  specific  hypothetical  missions.  By  analysis, 
define  a sensor  complement  to  carry  out  the  missions.  To  perform  this 
analysis,  consideration  must  be  given  to  target  type,  background  and  clutter, 
operating  environment,  sensor  resolution,  field(s)-of-view,  and  number  of 
members  in  the  crew  and  their  workload. 

Task  2 

Define  a multi-sensor  display  system  to  bring  together  the  outputs  of  the  sen- 
sors defined  above,  annotate  them  with  needed  symbology,  and  present  them 
in  the  format(s)  which  will  make  them  most  useful  to  the  aircrew.  This 
analysis  requires  reference  to  all  the  data  listed  above.  In  addition,  the 
flight  scenario  (day  or  night,  etc.  ),  restrictions  on  operation  (positive  visual 
identification  before  attack,  etc.  );  type  of  visual  access  required  (head  up  or 
head  down),  and  the  need  for  auxiliary  information  (attitude  or  air  data,  etc.  ) 
will  be  of  interest.  A significant  part  of  the  task  will  be  to  choose  those 
implementation  techniques  which  offer  the  greatest  opportunity  for  improve- 
ment over  existing  techniques  at  an  acceptable  cost. 

Task  3 

Fabricate  a demonstration  model  of  the  chosen  multi-function  display  and 
conduct  a laboratory  evaluation  of  its  performance.  Depending  on  resources 
available,  the  demonstration  model  may  be  designed  either  for  laboratory- 
only  analysis  or  for  an  eventual  flight  test.  The  latter  is  preferable,  though 
the  former  would  yield  information  valuable  in  designing  a flight  system. 
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CONCLUDING  REMARKS 

We  have  made  a number  of  tentative  recommendations.  We  feel  that  much 
more  work  is  required  in  both  research  and  development.  The  design  of 
cockpit  information  systems  has  been  of  great  importance,  and  the  develop- 
ment of  multifunctional  and  multisensor  displays  makes  this  task  even  more 
challenging. 
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APPENDIX  A 

LUMINANCE  AND  CONTP.AST  REQUIREMENTS 

Luminance  of  emissive  displays  is  reduced  if  the  active  elements  are  not  on  during 
the  0.  1 second  integration  time  of  the  eye.  Most  display  materials  are  quoted  for  their 
peak  luminance  unless  it  is  the  time-averaged  luminance  of  the  display  that  is  measured. 

It  is  necessary  to  know  the  refresh  rate  of  a display  when  it  is  multiplexed  or  matrix 
addressed  to  establish  the  relationship  between  the  required  peak  luminance  (Br)  and 
desired  time-averaged  luminance  (B). 

Br-S-^P  (A1) 

where 

N = number  of  refresh  periods  during  0.  1 second  integration  time  of  the  eye. 

L = number  of  characters  to  be  sequentially  multiplexed  or  the  number  of 
lines  of  a matrix  display  being  driven  one  line  at  a time. 

The  required  peak  luminance  must  be  170  times  the  time-average  of  brightness  for  a 
500- line  matrix  address  display  with  a 30  TV  frame  rate.  It  is  apparent  that  a display 
with  intrinsic  or  on-site  memory  would  greatly  improve  display  luminance  as  the  driven 
element  would  be  on  during  the  entire  refresh  period. 

The  luminance  of  a display  (instantaneous  threshold)  must  be  at  least  1/100  of  the 
luminance  level  of  the  surroundings  (to  which  the  eye  is  adapted)  to  be  visible.  However, 
this  data  was  from  test  of  a black  square  with  an  angular  subtend  against  a white  back- 
ground, and  a smaller  ratio  of  2/100  or  5/100  should  probably  be  used  for  a/c  displays 
Therefore,  an  aircraft  display  should  have  a luminance  of  from  680  to  1700  cd/m2to  be 
visible  with  the  eye  adapted  to  the  luminance  levels  of  white  clouds  (i.e.  , 34  000  cd/m2). 

The  intrinsic  contrast  between  on  and  off  elements  of  a display  are  degraded  when  that 
display  is  multiplexed.  The  relationship  of  the  required  display  intrinsic  contrast  ratio 
(Cr)  to  the  desired  time-averaged  contrast  ratio  (C)  is 

CR.£fi  <A2) 

The  intrinsic  contrast  between  cells  is  measured  when  the  display  element  is  on  over 
the  entire  integration  time  of  the  eye.  The  time-averaged  contrast  is  measured  while 
the  display  is  being  multiplexed  or  matrix  addressed  and  the  display  cell  is  not  on  con- 
tinuously." If  a time-averaged  contrast  of  10:1  is  required,  then  an  intrinsic,  contrast  of 
1700:1  is  required  for  a 500- line  matrix  display  with  a 30  frame  refresh  rate  (N  = 3.  0). 
Because  few  display  materials  have  this  high  of  an  intrinsic  contrast,  on-site  memory’  is 
needed  for  matrix  addressed  passive  displays. 

This  contrast  expression  (Equation  A2)  is  valid  for  a passive  reflective  display  under 
any  conditions  of  ambient  illumination  as  the  luminance  ratio  is  the  same  as  the  contrast 
ratio.  However,  the  contrast  of  an  active  display  is  degraded  by  any  ambient  illumination 
on  the  display  due  to  direct  sun  illuminance  or  ambient  illuminance  from  the  sky.  The 
time-averaged  contrast  (C)  of  an  active  display  is  also 

C = CR  L <A3) 
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where  Bj^  is  the  peak  cell  brightness  during  the  on  time,  and  Bt  Q is  the  average  cell 
brightness  during  the  off  time. 

The  time-averaged  contrast  ratio  changes  when  the  luminance  due  to  the  effective 
display  reflectance  (R)  with  an  incident  illumination  (E)  is  added  to  the  intrinsic  cell 
luminance  levels  B^jj  and  B^  0: 

(B„  ,+RE/tt)  , 


(Bj  Q + R E/tt) 


(B„.  + R E/tt) 


(BHi/CR  + R E/tt)  L 


(A4a) 


It  can  be  seen  that  the  time-averaged  contrast  is  reduced  by  an>  illuminance  on  the  display. 
The  required  (peak)  display  luminance  (Bj^)  during  the  cell  on  time  is  found  from  the 
above  equation  to  be 

n RE(N/I.  - C)  .... 

BIIi  ' 7 (C/C R - N/I.)  (A5> 

An  example  of  increased  display  luminance  requirements  is  given  here  for  explanation. 
Direct  sun  illuminance  is  about  100  000  lm  / m 2 , Ambient  illuminance  (Ep)  on  the 
display  from  white  cloud  luminance  (Bp)  over  a solid  angle  (0)  of  1.6  steraaians  (esti- 
mate of  solid  angle  of  white,  bright  clouds  through  cockpit  windows)  is  approximated  as 


,,  _ 1,  6 x 34  000  cd/m 

EC 7T — 

Ec  = 28  000  lm  / m 2 

A nonmultiplex  display  (N/L  = 1.  0)  in  direct  sunlight  with  an  intrinsic  contrast  ratio 
(CR)  of  100:1,  a desired  display  contrast  (C)  of  10  with  a cell  reflectance  of  0.  05  would 
require  an  intrinsic  peak  luminance  (Bj^)  of 

~ . RE  (1  - C)  

BHi  • r (C!Cn  - 1)  (A7) 

In  substituting  for  the  case  of  direct  sun  illumination, 

0.  05  x 105  (1  - 10) 

Hi  ' - (10/100  - 1) 

2 o 

BHi  = 16,  OOOcd/m  and  in  cloud  ambient  illumination,  = 4 456  cd/m  . 


It  should  be  noted  that  as  the  intrinsic  contrast  (Cpj)  approaches  the  required  contrast 
(C)  or  when  C/Cn  becomes  unity,  the  display  luminance  requirement  becomes  infinite. 

It  can  be  concluded  that  the  intrinsic  contrast  of  a nonmultiplexed  display  should  be  roughly 
ten  times  the  desired  display  contrast  to  avoid  unreasonable  display  brightness  require- 
ments in  an  aircraft. 

The  display  luminance  and  intrinsic  contrast  ratio  requirements  for  a matrix  line- 
at-a-time  display  becomes  severe  in  the  ambient  illumination  of  an  aircraft  cockpit.  The 
required  peak  display  intrinsic  luminance  is 

n . ._RK  (X/L  - C) 

BHi  ' - (C  'C  ' - \/U  (Ao) 

K 

The  numerator  is  negative  as  N/T.  is  less  than  C for  matrix  displays.  Therefore,  C/Cn 
must  be  less  than  N/L  to  have  a positive  (real)  solution  for  the  display  peak  luminance. 

If  C/Cjj  is  to  be  10  times  less  than  N/L  to  minimize  the  peak  luminance  requirements, 
then  the  intrinsic  contrast  (CR)  must  be  17  000;  1.  This  is  for  the  matrix  display  case  for 
C = 10,  L = 500  lines,  and  N = 3. 

CH  > 10  ^ (A9) 

o ^ 10x10x5  00 


CR  > 16  660:1 

This  high  contrast  is  expected  because  matrix  address  ceil  elements  are  on  for  a 
short  time,  making  residual  luminance  of  the  "off"  cells  significant.  Therefore,  it  is 
critical  that  there  be  very  little  drive  leakage  to  the  off  cells,  i.e.  , the  off  cells  residual 
luminance  must  be  17  000  times  less  than  the  peak  luminance  of  the  on  cell.  The  peak 
display  luminance  requirement  for  this  case  in  direct  sunlight  is 


- 0.  05  x 10  (3/500 


(A10) 


6 2 

B„.  = 3.  0 x 10  cd/m  for  direct  sun  luminance 
6 2 

Br.  =8.2x10  cd/m  “ for  ambient  sky  luminance 

These  luminance  levels  are  very  high  compared  to  what  is  available  for  flat  panel 
materials.  The  best  solution  for  matrix  displays  in  an  aircraft  is  to  develop  on-site 
intrinsic  or  inherent  memory  capability;  it  can  be  concluded  that  development  of  on-site 
memory  is  a firm  requirement  for  active  matrix  addressed  displav-  in  an  aircraft. 

This  analysis  does  not  account  for  the  improvement  of  displav  contrast  and  reduction 
of  display  brightness  through  the  use  of  ambient  light  suppression  lace  plai.es,  louvers, 
or  filters  because  these  devices  will  probably  not  reduce  the  brightness  or  contrast 
requirements  of  a nonmemory  active  displav  to  a reasonable  level.  These  device.-, 
however,  will  permit  some  active  nonmultiplex  displays  or  a matrix  displav  with  on-  in 
memorv  to  function  in  ambient  illumination.  Operation  in  direct  sun  ma\  be  po  siblc 
only  for  a passive  display  in  the  nonmultiplex  mode. 
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